I gave a presentation recently at UK Museums on the Web entitled “The Intertubes Everywhere”. It was a re-working of my Ignite Cardiff talk, with a gentle angle towards cultural heritage. Here are the slides:

The one-liner for those that don’t have the time to go through the slides is something like this: I believe that although mobile has been held up as THE NEXT BIG THING for some time, we are reaching a kind of “perfect storm” of conditions where it is at last becoming a viable reality for many users and therefore something for institutions to think about, too.

This is as much to do with effective marketing and consciousness raising as it is to do with device or network capability: if you’ve tried buying a mobile phone in the last year or two, you will have been offered mobile internet; if you go to a mobile phone company website today, you’ll see smartphones, dongles and internet on the go on their homepage. It would be very hard to miss this kind of marketing push. Couple this with the radical improvement of mobile content, the beginnings of location-based services and the increasing speeds and capability of a “normal” mobile device, and it seems pretty clear that we’re on the cusp of something pretty big.

If you’re in any doubt, check out slides 25-35 of the presentation that Dan Zambonini and I did at DISH 2009, which have some interesting figures on changing mobile usage. With device replacements happening on average every 14 months, even the old-school phones that don’t support mobile internet won’t be here for much longer.

With this level of exposure, it’s obvious that museums and other cultural heritage institutions are going to be following along and getting excited about mobile, either building iPhone apps or creating mobile versions of their sites.

While it is excellent to see innovation in this field, I’m slightly underwhelmed by some of the mobile offerings starting to appear that seem to be more “because we can” rather than “because we should”, in particular the current trend (and I’m deliberately not giving any examples – you can go find them yourself!) for “mobile collections search”.

It seems to me that the single mantra which should surround any mobile web development project right from the start is something like “never forget: the mobile browsing experience is far, far inferior to the desktop browsing experience”.

Browsing a mobile website is generally not a fun time. You don’t relax when you’re browsing on a mobile; you don’t lose yourself in the content: you’re there in sit forward mode, and you want to do one of two things:

1. find some information and get out as quickly as you can
2. use the capability of the “mobile” bit of the experience to do something…well, “mobile”

The first point is a no-brainer, IMO. Consider when and how I might choose to browse a museum website on my mobile. The answer is not “in my living room at home” – if I’m there, I’ll go find my laptop and have a far easier and more pleasurable experience in sit back mode. The answer probably is (and don’t shout at me for being obvious..) but when I’m mobile. I’m out and about, wondering what to do at lunchtime, thinking about whether a museum is open or where I can get tickets or how to get there. I’m not on WIFI, and I want the information as quickly and as seamlessly as possible. I don’t want images, I don’t want interaction, I want information. And I want it right now. And – this is the painful bit – I really, really don’t want to browse the collections. Why would I want a second-rate experience of browsing content using a 2″ screen, some clumsy non-mouse interaction touchpoints and a slow connection? And – more to the point – why would I possibly want to stand in the street (being mobile…) and look at museum collections? I don’t*.

* Actually, sometimes I do, provided the mobile experience adds something. And this is where point 2 comes in:

If I can have an experience which augments my real experience rather than just providing a poor quality facsimile of an online experience - then you’re talking about truly putting mobile capability to good use.

So for example – if I’ve got a known location (and this can mean GPS but more likely in our museum context means “I’m standing in front of artefact X and my phone knows that because I’ve keyed in something to tell it this”), then now is the time for the museum to give me additional information about other similar exhibits, let me bookmark that artwork, or share it with my network.

mobile.nmsi.ac.uk - something I knocked out about 5 years ago and still live!

Some of the museum sites we’re starting to see are making use of this capability – check out BlkynMuse on your mobile (and note the immediate emphasis on “where are you on-gallery?”) as a good example; but there also seems to be an increasing number who are simply putting their museum collections online as they are in some kind of mobile format – either a mobile optimised site or (worse) an iPhone application, with none of the context-sensitivity that makes mobile a value-add proposition for end-users.

Much as I’m glad to see innovation in this space, I’d much rather see museums focussing on point 1 above by having a mobile-sniffing code on their homepage and redirecting to an optimised m.museumsite.com page with visiting information, than putting in a huge amount of effort into providing mobile-optimised collections search. At the very worst, museums should have the subdomain m.*** or mobile.*** and there have a script to strip out the images and so on. There are many ways to do this – here, for example is the Museum of London site stripped using a simple PHP script from Phonefier, or see these tips on how to create simple “mobilised” versions of your existing site with zero extra effort.

Once the simple and high-gain win is done, then it’d be great to see some location-specific and innovative approaches to “virtually collecting” or augmenting collections experiences. But the “browse our mobile collections site” without really thinking about the use-case is pretty much saying: “go here on your mobile and you can have an experience which is infinitely worse than the one on your desktop with absolutely no upside”. In other words, no thanks.

What do you think? Has your museum got a mobile site for visitors, or just for collections, or none at all? What mobile apps have you downloaded or accessed that provide museum collections (or other) information? How was it for you?

UPDATE (about 3 minutes after I posted this…): I just realised I utterly neglected to talk about gaming. Which, IMO, is where mobile (and in particular mobile collections) have a huge amount of potential. I think this’ll have to wait for a future post

Last week was the annual UK Museums on the Web conference.

Things were particularly hectic and exciting for me this year for a whole host of reasons:

• We launched a new MCG website in the week before the conference - this was a full migration to WordPress MU which I’ll write more about shortly;
• We were working behind the scenes with the excellent Laura Kalbag to develop a new logo and design guidelines for the group which we also needed to get live by conference day;
• I suggested I build and trial a QR tag demo based on individualised badges (more below..);
• I was giving a presentation on ubiquitous technology…

Crazy busyness aside, overall this was for me the best UKMW conference yet, for one very simple reason: we’d managed to get a range of speakers from outside the sector. Often, domain-specific conferences have a tendency to focus inwards, and although it is incredibly useful to see projects that are specific to that sector, I think it is as important that everyone keeps an eye on the outside world. This is particularly the case right now, as museums come down off the 2.0 peak and start to ask where the value is and how best to capitalise on an ever-decreasing budget.

Many other people have done a much better job of describing in detail who talked about what at the conference. If you’re interested, see the many things tagged ukmw09 on Google Blog Search. For me it was probably Paul Golding, Andy Ramsden or Denise Drake who did the most insightful talks for me, but actually every presentation was really interesting and led to a fascinating day.

So now to the point of this post: the beta “onetag” system I put in place to allow delegates to use QR tags in a “real-world” scenario.

For those who aren’t familiar with QR codes, I’d suggest a brief moment over on Wikipedia or go with the one-line description: “barcodes for linking the real and virtual worlds”.

As well as wanting to give people a QR example to play with, I based the idea for the system on a problem which I think needs solving, particularly at conferences: business cards are irritating, wasteful and require re-keying (hence duplication) of details. The idea, therefore, was to give everyone at the conference a personalised badge with the QR code on it, get them on-board prior to the event so that as many as possible had QR code readers installed on their mobiles, and then sit back and watch how this kind of system might be used, or not!

For the badges, I used local print firm Ripe Digital, who are not only incredibly helpful but also have the ability to run what is essentially a large-scale mail-merge: I designed an A6 badge in Adobe Illustrator which had various fields in it which were populated from an Excel spreadsheet of delegates. (Incidentally, we’d made extensive use of Google Docs during the conference for gathering and munging delegate names, and this really paid off in terms of sharing, collaborating and processing delegate information).

I created the actual codes using a fairly nasty mix of Google Charts, downthemall and mailmerge (don’t ask) – once I’d got a local folder with all 100 or so QR codes in it, I just referenced those codes in the AI document and asked Ripe to insert the specific QR tag at top right of the printed badge.

Here’s the front of my badge – note (important, this) that the grey code under the QR tag is also unique to each person, allowing those without QR readers to take part in the demo as well.

The badge, incidentally, also contained sponsor information and outline timings for the day on the back, a detailed description of the timings and speakers on the inside fold and a delegate list on the reverse. The badge was folded from a single sheet of A4 into an A6 wallet hung on a lanyard around people’s necks. The basic premise was to save as much as possible on enormous (mostly unwanted) wads of printed material and focus instead on the key information that delegates are likely to want.

Assuming (not a great assumption, but go with me for now) that someone not only had a reader installed on their mobile but also managed to successfully read the code, here’s what happened:

The very first time a delegate uses the app, they get directed to a mobile-formatted web page which asks them for their PIN (QR number) details – that’s the bit in grey under their glyph:

Delegates only had to do this once (I placed a cookie to keep the logged-in state) – once they had, and on all future taggings, they get redirected to a screen showing them details for the person they just tagged:

This is only so much use, especially given the name badge itself has all of this detail on it already, so I also built in the functionality behind the scenes to email the “taggees” details to the “tagger”, both as a plain email but also with an attached vCard. This therefore means that the person who did the tagging can easily add this contact to their address book without having to re-key any of the information. Here’s how the email looks in Outlook:

And that, basically, is that –

So did people use it? And if so, how?

Behind the scenes, I was grabbing some data each time anyone carried out a tagging. The data I intended to capture was: who did the tagging, who they tagged and when. As it happens, and annoyingly, my script failed on the “when” bit. I also realise that in hindsight I really should have captured the user agent for each tagging as well – then I would have some insight into what people used, most common devices, etc. With a fair amount more time (of which I currently have none!) I could probably marry up the server logs with device types, but for now I’ll leave that bit of information to one side.

The first bit of interesting information is this: there were 81 taggings during the day, which was actually much higher than I’d anticipated.

27 different people used the system (out of around 100 registered delegates)

On average, the people who did tag someone did it on average 3 times, although this figure is skewed upwards by one person who tagged 21 people! Here’s how the distribution looks:

Another view on this data shows that a fair number of people also tagged themselves, presumably to familiarise themselves with the software (that’s the visible diagonal line bottom left to top right):

So what did we actually learn from this: first of all, total simplicity from a user perspective is – as always – absolutely key. Here, we had a willing audience who had been given a heads-up to expect to install the software, definitely would have been “geek skewed” in terms of internet-enabled devices and were willing to play; and although I was pleased that lots of people took part, the figures show that this is clearly far from being a “everyone does it” activity.

Secondly, the blocker – again, as always – wasn’t just the technology but the social issues that surrounded the technology. I saw lots of people tagging, but this wasn’t an “invisible” activity of the type that makes for seamless interaction. People had to stop other people, ask them to hold still, take a photo, wait for the software to catch up, try again when the barcode failed to read and so-on. However hard I tried to make the back-end seamless, QR software just isn’t good enough (yet) to deal with quick shots, moving targets, wobbling hands. In this particular instance (and this is actually the next stage of onetag that I’m going to look at), RFID or SMS based tagging would have been slicker.

Thirdly, although I see business cards as an issue, it isn’t necessarily a problem which is identified as such for everyone. Exchanging a business card is natural; scanning a badge isn’t. So for this to really work, the technology either needs to be invisible (I just wave my reader over your badge, no focussing or waiting or holding still..) OR the win needs to be much more tangible (a tagger gets more information about a tagee, or there is some kind of other incentive to make the connection, etc). Providing more information obviously has privacy issues, and also potentially usability issues; as the incentive becomes bigger, so – normally – would the complexity of both the system and the explanations underlying that system.

Overall, I was very pleased with how the system worked, and also delighted that so many people took the time to test it out – so thanks to you, whoever you were!

I’m going to be continuing to develop the various onetag systems, and am always up for hearing from you if you’d like me to put something together for your conference or event – just comment or email and I’ll get in touch.

I’m absolutely delighted (and only slightly scared) to announce that I’ve been commissioned to write a book for Facet Publishing.

Ever since I started working with museums online, I’ve felt that there is a need for strategic advice to help managers of cultural heritage web presences. There are of course hundreds of thousands of resources if you’ve got technical questions, but not many places where you can ask things like “how should I build my web team and structure my budget?” or “how do I write a strategy or business plan?”.

Facet approached me in July asking whether I’d be interested in authoring something for them, and this seemed like the ideal opportunity to try and answer some of these questions.

My (draft) synposis is as follows:

This book will provide a guide for anyone looking to build or maintain a cultural heritage web presence. It will aim to cater both to those who are single-handedly trying to keep their site running on limited budget and time as well as those who have big teams, large budgets and time to spend.

As well as describing the strategic approaches which are required to develop a successful online presence, the book will contain data and case studies on current practice from large and small cultural heritage institutions. This research will help give the reader an insight into how these institutions manage their websites as well as providing hints and tips on best practice. It will have an accompanying web presence which will provide template downloads and other up-to-date information including links and white papers.

As you’ll see, I have no intention of trying to do this all by myself – over the coming year I’m going to be on the phone to many of you (hide now!) asking how you do what you do, and compiling this into what I hope will be a useful guide.

If you have any ideas about what I should include, or the questions I should be asking – please do get in touch either via this blog or on Twitter at @m1ke_ellis!

I’m delighted to announce the beginning of what I hope will be an exciting (and useful!) mini-project.

Museuminaday is a concept which Dan Zambonini and I have come up with to support our workshop “The Lightweight Museum” at the DISH conference in December.

Hopefully the name should do most of the work in explaining what museuminaday is about: we intend to build a museum website in 12 hours, start to finish, documenting the techniques we use and the things we discover along the way.

Most of the work will happen during a single day (2nd November 2009 – and we’ll be filming and live-blogging on that day) but we’re also documenting everything we do before then and taking time off our 12 hour deadline as we go. You can see our public Google Spreadsheet which outlines everything we’ve spent time and money on to-date.

We hope we’re going to succeed, but we’re more than ready to fail, too – either way we hope that we’ll bring something useful to the table.

You can read more about the project on the about page – or just follow us on Twitter at @museuminaday. Comments, suggestions, ideas – more than welcome!

Let’s ask this: Just what do museum website users want?

Actually, before we do that, the biggest question is “who is our audience?”.

Wait. Before we do that, let’s assume that – what – 70-80% of museum website users want to find out some logistical stuff: “what’s on? how do I get there? how much is it?”. Let’s assume that this bit is solved with a page or two of dull but useful information. Let’s ignore the 70-80%. They’re boring. There’s only so much you can do with a map and some opening times, right?

Now let’s consider the other stuff – the content – the collections, the exhibition stories, the richness. Just who are these people, what do they want, and where do they come from?

Determining audiences for museum websites is a slippery game which generally involves phrases like “lifelong learners” (everyone) or “educators” (teachers, parents, children – oh wait, everyone) or just “everyone”.

I’m being slightly mean, and actually the definitions are a little bit better than that, but still there is an underlying tension which is something to do with deeper questions about success, publicity, depth of resources, marketing, integrity – and that horrible, horrible phrase which frequently does the rounds: dumbing down.

When a curator oversees a website, for instance, he or she often fights the dumbing down thing tooth and nail. Curators are about depth, about academic rigour and cleverness. Curators aren’t (often) about publicity, traffic, sound-bites and volume. This is fine, and museums should be about quality and richness and integrity. If it wasn’t for this, they wouldn’t be the respected institutions that they have become.

The problem is that museums online want (and increasingly need) to be mainstream, too. We see Flickr and Facebook and Google and viral marketing and Twitter and….[etc] and, frankly, we want some ‘o’ that. And the tension there becomes more intense. Can you build traffic and volume and virality online and still manage to “not be dumb”? Can these deep, rich, academically sound experiences also be mainstream? Is – getting to the crux of the question – a mainstream user shallow or deep?

One of the big, enduring discussions, for example, is about how Google provides search into museum collections. Museum people tend to twitch if you suggest they should focus on exposing their collections sites to SEO best principles and forget the in-house search (or even just stick their stuff on Wikipedia and forget the whole in-house piece altogether), because they say that Google doesn’t provide the granularity that is required. For some researchers – those who want to find out the year an object was invented or the country of origin, for example – this lack of granularity is indeed a problem. For many others – those who just want a picture of any old steam engine for their desktop or wherever – it isn’t.

Balancing this requirement / audience / success equation is in itself a game. The best solution (do both) is clearly the answer, but many institutions fail to realise this, tending to focus on arcane in-house terms and interfaces rather than trying to find ways of building SEO via common content entrance points like Google. It becomes a user interface question, yes, but it is also about much bigger-picture strategic issues about success.

What each museum needs to decide is what this success looks like. And if – as is usually the case – success is about museums becoming more used, more embedded in people’s lives, more human – then success is, frankly, about Google. There, I said it. Where else does anyone begin a search for – well, anything? Do we really think that people come to museums to begin their search? Really?

So success – in the case of Europeana, for example – seems to me to be about asking the question: “can I find Europeana stuff on Google?”, not “can I find Europeana stuff on Europeana?”. When I’m looking for information on Leopold Mozart, I’m not – ever – going to start my search on one of our individual museum sites or any of the aggregators, federators or whotsitators that have been developed, including Europeana. I’m going to Google. Firstly, because I clearly don’t know who knows stuff on Mozart’s father and I can’t go there if I don’t have that specialised bit of information yet (and Google (currently) provides the single best starting point for my query); but secondly, because Google is there as my homepage, a hook in my Chrome browser search bar and as a known entity in my consciousness. Why would I start my search looking at detail in a single book when I’ve got access to general information about the whole library?

This is grandmother / eggs for many people working in museums, but I’m not sure it is as obvious to the big projects we’ve seen emerging from the museum sector. For some of these projects, specialised audiences are their success, in which case local approaches do work better. But for the majority, success is increasingly about making enough SEO noise for more general audiences.

And is this “dumbing down”? Yes, I suspect it probably is.

I first joined Twitter in 2007. In fact, if www.whendidyoujointwitter.com is correct, I joined on 20th February 2007.

My first account was @dmje. I tweeted in that way that everyone seems to first tweet – a sporadic few “just what the hell is this Twitter thing all about?” followed by a long gap, followed by a re-emergence as more people I knew found themselves on it. I also, of course, blogged (“All Noise, No Signal“) and have been slowly eating my words (some of them, not all) ever since.

For a long time, my @dmje account worked well. But after a while, I started to become very aware that the person that I am (opinionated, personal, direct, a little bit sweary..) was different from the person I either *should* be or was somehow expected to be (professional, supportive, focused).

At that point in time – in fact, prompted by a slightly sweaty moment in which I tweeted a few bits and bobs which I probably shouldn’t have from a professional perspective – I decided to make @dmje a private account and create a new public persona, @m1ke_ellis. Again, according to whendidyoujointwitter, this happened on 22nd May 2009.

I went through a fairly painful process of moving across *some* contacts to my m1ke_ellis account but leaving others at @dmje. My criteria? Very, very loose, but broadly based around: “If we’ve met and drunk a beer together then @dmje, otherwise @m1ke_ellis…”. There are exceptions to this rule, though. Obviously

I’m now maybe 5 months down the line, and I’m still not entirely happy with the outcome; although each time I think about the possible alternatives I always come back to what I’ve done as being the best way, albeit far from perfect.

Here’s the thinking:

The good:

• I can continue to rant, unabridged and privately (except, obviously, to a group of trusted and known personal friends) using my @dmje account. I use this account far more than my public one (sadly, nearly 10,000 tweets…)
• I follow about 120 people, I have about 110 people who I’ve allowed to follow me. These people are real to me. In true Dunbar style, I see my Twitter stream for @dmje and feel a personal connection with each and every person on that list.
• …I can therefore cope with the quantity and noise
• Tweets to and from the @dmje account are much more conversational, much less “broadcast”
• I can retain a “professional” persona at @m1ke_ellis, tweeting about work and technology related stuff. This is particularly useful at conferences and so on
• Having a public account of some description is useful when it comes to feeding a stream to blogs, profile, and so on

The bad:

• By far and away the single worst thing about this approach is this: I’m not two people, and although this can sometimes get ugly (yes, I ranted about Creative Spaces; no, I wasn’t particularly “professional”, but I feel passionate about some things..)
• From a marketeers perspective (and I don’t subscribe to this viewpoint at all, btw), I’ve done A Bad Thing by splitting my Twitter accounts. While I’ve watched some people moving up to thousands of followers, I’ve split my juice (urg!) across 2 accounts. Actually, more – I also use @bathcamp and @eduserv for other specific purposes. If I was after followers (I’m not), I should probably have stuck with a single “me” account.
• Maintaing two or more accounts is challenging, logistically. Although Tweetdeck (my preferred desktop client) and EchoFon (mobile) both support multiple accounts now, it is very easy to tweet the wrong thing to the wrong account. More to the point, it is hard to maintain momentum with an account if your attention isn’t on it all the time

There is a deeper point to all this: Embracing social media requires a fairly complex understanding of personality and tone of voice. I might be a more professional me over at @m1ke_ellis, but how is that me different to the me at @dmje? You’re not likely to hear about my kids, my wife, my life, my hangovers, the gig I just went to, the #bus14 journey I nearly got killed on.

But there again, if you’re listening to the professional me then you probably don’t want to hear that anyway, right? Or do you? How real is the me who just talks about work? Not very, in one sense, because my family and that other stuff is (obviously) waaaay more important than my working life. And it’s not like I can effectively split my interests in that way. I live and breathe web stuff – this is far from being a day job for me.

Actually, I think the most successful social media people and companies manage to balance this rather better than I have. Take @andypowe11 for example. He’s public and not only tweets about metadata and work stuff but also rants on occasion, too. He’s got better self control than me (he’s as rude, but swears less..), but still.

I don’t like Twitter as broadcast mechanism, and I think naturally once you pass a level of followers/followees that is what it becomes, unless you’re on top of it all of the time. Personally I dislike it when I “@” someone and they don’t reply; clearly someone with thousands of followers is unlikely to respond all of the time. Twitter then moves from being a conversation to being something different, a something which I feel doesn’t carry the personality which social media perhaps should.

There’s acres and acres of stuff to read and write about the whole National Portrait Gallery legal action threat against Wikimedia contributor Dcoetzee and his addition to the Wikimedia collection. I’m not going to try and add to the noise too much but it would seem apposite to at least comment given my current thread of presentations and posts is all about freedom, openness and MRD.

As always (just like the argument currently brewing about Free), there are two possible dangers in any debate like this. First, we go into too much detail and lose the view of the house because we’re examining the bricks too closely. Second, we polarise the debate.

I’m good at polarising, being a bear of simple brain – particularly when it comes to copyright. Simply, I don’t think it works in many cases, and I think this particular example holds – on many levels – great reasons as to why not. Cross-country, cross-domain, cross-sector, hidden images, non-hidden images, etc etc. This level of complexity doesn’t hold well with users, and they will abuse, either knowingly or unknowingly.

Having said that, there are clearly two sides to this particular debate, and actually I think both sides are being pretty reasonable. NPG have offered medium sized pictures; Wikimedia has been on the case for some years seeking access to these (arguably) public domain images. The discussion over the detail in this particular case will ramble on; the legal threat will be sorted out of court; everyone will ultimately go away at least semi-happy.

The bigger picture is the more important question, and it is this: why are cultural institutions putting collection (images) online? I ask this as an open question, as un-loaded as it can be (given you probably know where I’m coming from on this).

The possible answers are these (none is mutually exclusive, by the way):

• to sell them / variations of them, such as prints, etc
• to increase exposure to them
• to increase exposure to the holding institution
• to increase ticket sales / physical visits to the holding institution

So with these in mind, I think the important questions in this particular debate are not about the devil detail of cross-country copyright or whether Dcoetzee “should” have done what he did. I think they are:

• does the exposure on Wikimedia increase exposure? (Answer: yes)
• does exposure of hi-res pictures stop people from buying them (Answer: unknown, but possibly not)
• does the exposure of the images improve the standing of the institution (as being a place that “has a great collection”) ? (Answer: yes)
• does the exposure of the images increase click-through to the NPG website (and hence, assuming at least some kind of connection between traffic and physical visits) ? (Answer: unknown – I’m about to submit a FOI request to see if we can find out, but probably yes)
• does the threat of legal action make NPG look good? (Answer: not really)

There’s some great questions here, which I’ve been asking our sector to answer for a while. Where is value in a networked age? How does virtual equate to physical? Does exposure increase or decrease physical sales (go ask Anderson or Gladwell this one…).

Just as a closing thought, I wonder if the NPG will be chasing Yahoo! for this YQL query or Google Images for this one? I suspect not.

The week before last (30th June – 1st July 2009), I was at the JISC Digital Content Conference having been asked to take part in one of their parallel sessions.

I thought I’d use the session to talk about something I’m increasingly interested in – the shifting of the message about machine readable data (think API’s, RSS, OpenSearch, Microformats, LinkedData, etc) from the world of geek to the world of non-geek.

My slides are here:

Here’s where I’m at: I think that MRD (That’s Machine Readable Data – I couldn’t seem to find a better term..) is probably about as important as it gets. It underpins an entire approach to content which is flexible, powerful and open. It embodies notions of freely moving data, it encourages innovation and visualisation. It is also not nearly as hard as it appears – or doesn’t have to be.

In the world of the geek (that’s a world I dip into long enough to see the potential before heading back out here into the sun), the proponents of MRD are many and passionate. Find me a Web2.0 application without an API (or one “on the development road-map”) and I’ll find you a pretty unusual company.

These people don’t need preaching at. They’re there, lined up, building apps for Twitter (to the tune of 10x the traffic which visits twitter.com), developing a huge array of services and visualisations, graphs, maps, inputs and outputs.

The problem isn’t the geeks. The problem is that MRD needs to move beyond the realm of the geek and into the realm of the content owner, the budget holder, the strategist, for these technologies to become truly embedded. We need to have copyright holders and funders lined up at the start of the project, prepared for the fact that our content will be delivered through multiple access routes, across unspecified timespans and to unknown devices. We need our specifications to be focused on re-purposing, not on single-point delivery. We need solution providers delivering software with web API’s built in. We need to be prepared for a world in which no-one visits our websites any more, instead picking, choosing and mixing our content from externally syndicated channels.

In short, we now need the relevant people evangelising about the MRD approach.

Geeks have done this well so far, but now they need help. Try searching on “ROI for API’s” (or any combination thereof) and you’ll find almost nothing – very little evidence outlining how much API’s cost to implement, what cost savings you are likely to see from them; how they reduce content development time; few guidelines on how to deal with syndicated content copyright issues.

Partly, this knowledge gap is because many of the technologies we’re talking about are still quite young. But a lot of the problem is about the communication of technology, the divided worlds that Nick Poole (Collections Trust) speaks about. This was the core of my presentation: ten reasons why MRD is important, from the perspective of a non-geek (links go to relevant slides and examples in the slide deck):

1. Content is still king
2. Re-use is not just good, it’s essential
3. “Wouldn’t it be great if…”: Life is easier when everyone can get at your data
4. Content development is cheaper
5. Things get more visual
6. Take content to users, not users to content (“If you build it, they probably won’t come”)
7. It doesn’t have to be hard
8. You can’t hide your content
9. We really is bigger and better than me
10. Traffic

All this is is a starter for ten. Bigger, better and more informed people than me probably have another hundred reasons why MRD is a good idea. I think this knowledge may be there – we just need to surface and collect it so that more (of the right) people can benefit from these approaches.

I just finished my talk at Mashed Library 2009 – an event for librarians wanting to mash and mix their data. My talk was almost definitely a bit overwhelming, judging by the backchannel, so I thought I’d bang out a quick blog post to try and help those I managed to confuse.

My talk was entitled “Scraping, Scripting and Hacking your way to API-less data”, and intended to give a high-level overview of some of the techniques that can be used to “get at data” on the web when the “nice” options of feeds and API’s aren’t available to you.

The context of the talk was this: almost everything we’re talking about with regard to mashups, visualisations and so on relies on data being available to us. In the cutting edge of Web2 apps, everything has got an API, a feed, a developer community. In the world of museums, libraries and government, this just isn’t the case. Data is usually held on-page as html (xhtml if we’re lucky), and programmatic access is nowhere to be found. If we want to use that data, we need to find other ways to get at it.

My slides are here:

A few people asked that I provide the URLs I mentioned together with a bit of context. Many of the slides above have links to examples, but here’s a simple list for those who’d prefer that:

• http://hoard.it as an example of “intelligent scraping” being used to take on-page content and re-deliver it as “nicer” machine-accessible content
• Yahoo! Pipes being used to scrape segments of this page using the Fetch Page module
• Google Docs being used to scrape this page using the importHTML() function (see Tony Hirst’s excellent blog post for a better example)
• Dapper being used to scrape pages with this shape and display extracted data
• YQL being used to scrape this page and deliver search results into a REST query
• HtTrack for downloading entire websites / sections of websites
• RegEx visual tool
• HTML Tidy, a tool for cleaning up “bad” html, available as both a download and a COM object for use in your scripts
• Using OpenCalais for doing natural language text parsing (example form here)
• Yahoo! Term Extraction – example form here
• Yahoo! Geo
• Freedom of Information (example from Frankie Roberto), OCR (from me) and Amazon Mechanical Turk

Phew. Now I can see why it was slightly overwhelming

Watching Google’s amazing “what is a browser” video (below) it is easy (and I can almost hear the geeks laughing) to assume that these are just stupid people on a bad day. I mean, what the hell is wrong with them? “My browser is Google”? WTF?

The thing is, these aren’t stupid people. They’re just normal people, going about their normal lives doing normal things. And these are the people we’re building websites and interactive experiences for.

The phrase Problem Exists Between Chair And Keyboard (and wow, isn’t it interesting that there are LOADS of phrases on the same page which mean the same thing, and all equally rude..) was invented by developers trying hard to find excuses for the poor implementation they just rolled out.

The thing is, the problem isn’t BCAK, it’s In The Dev Team. Maybe we should invent a new acronym: PEITDT…

Longer term, this is of course more to do with tech literacy, being a digital native, familiarity with the web and so on. Shorter term, until we solve the literacy problem, we need to pay extra-special attention to users. And maybe never, ever say the phrase PEBCAK or any of its permutations again…