There’s acres and acres of stuff to read and write about the whole National Portrait Gallery legal action threat against Wikimedia contributor Dcoetzee and his addition to the Wikimedia collection. I’m not going to try and add to the noise too much but it would seem apposite to at least comment given my current thread of presentations and posts is all about freedom, openness and MRD.

As always (just like the argument currently brewing about Free), there are two possible dangers in any debate like this. First, we go into too much detail and lose the view of the house because we’re examining the bricks too closely. Second, we polarise the debate.

I’m good at polarising, being a bear of simple brain – particularly when it comes to copyright. Simply, I don’t think it works in many cases, and I think this particular example holds – on many levels – great reasons as to why not. Cross-country, cross-domain, cross-sector, hidden images, non-hidden images, etc etc. This level of complexity doesn’t hold well with users, and they will abuse, either knowingly or unknowingly.

Having said that, there are clearly two sides to this particular debate, and actually I think both sides are being pretty reasonable. NPG have offered medium sized pictures; Wikimedia has been on the case for some years seeking access to these (arguably) public domain images. The discussion over the detail in this particular case will ramble on; the legal threat will be sorted out of court; everyone will ultimately go away at least semi-happy.

The bigger picture is the more important question, and it is this: why are cultural institutions putting collection (images) online? I ask this as an open question, as un-loaded as it can be (given you probably know where I’m coming from on this).

The possible answers are these (none is mutually exclusive, by the way):

• to sell them / variations of them, such as prints, etc
• to increase exposure to them
• to increase exposure to the holding institution
• to increase ticket sales / physical visits to the holding institution

So with these in mind, I think the important questions in this particular debate are not about the devil detail of cross-country copyright or whether Dcoetzee “should” have done what he did. I think they are:

• does the exposure on Wikimedia increase exposure? (Answer: yes)
• does exposure of hi-res pictures stop people from buying them (Answer: unknown, but possibly not)
• does the exposure of the images improve the standing of the institution (as being a place that “has a great collection”) ? (Answer: yes)
• does the exposure of the images increase click-through to the NPG website (and hence, assuming at least some kind of connection between traffic and physical visits) ? (Answer: unknown – I’m about to submit a FOI request to see if we can find out, but probably yes)
• does the threat of legal action make NPG look good? (Answer: not really)

There’s some great questions here, which I’ve been asking our sector to answer for a while. Where is value in a networked age? How does virtual equate to physical? Does exposure increase or decrease physical sales (go ask Anderson or Gladwell this one…).

Just as a closing thought, I wonder if the NPG will be chasing Yahoo! for this YQL query or Google Images for this one? I suspect not.

The week before last (30th June – 1st July 2009), I was at the JISC Digital Content Conference having been asked to take part in one of their parallel sessions.

I thought I’d use the session to talk about something I’m increasingly interested in – the shifting of the message about machine readable data (think API’s, RSS, OpenSearch, Microformats, LinkedData, etc) from the world of geek to the world of non-geek.

My slides are here:

Here’s where I’m at: I think that MRD (That’s Machine Readable Data – I couldn’t seem to find a better term..) is probably about as important as it gets. It underpins an entire approach to content which is flexible, powerful and open. It embodies notions of freely moving data, it encourages innovation and visualisation. It is also not nearly as hard as it appears – or doesn’t have to be.

In the world of the geek (that’s a world I dip into long enough to see the potential before heading back out here into the sun), the proponents of MRD are many and passionate. Find me a Web2.0 application without an API (or one “on the development road-map”) and I’ll find you a pretty unusual company.

These people don’t need preaching at. They’re there, lined up, building apps for Twitter (to the tune of 10x the traffic which visits twitter.com), developing a huge array of services and visualisations, graphs, maps, inputs and outputs.

The problem isn’t the geeks. The problem is that MRD needs to move beyond the realm of the geek and into the realm of the content owner, the budget holder, the strategist, for these technologies to become truly embedded. We need to have copyright holders and funders lined up at the start of the project, prepared for the fact that our content will be delivered through multiple access routes, across unspecified timespans and to unknown devices. We need our specifications to be focused on re-purposing, not on single-point delivery. We need solution providers delivering software with web API’s built in. We need to be prepared for a world in which no-one visits our websites any more, instead picking, choosing and mixing our content from externally syndicated channels.

In short, we now need the relevant people evangelising about the MRD approach.

Geeks have done this well so far, but now they need help. Try searching on “ROI for API’s” (or any combination thereof) and you’ll find almost nothing – very little evidence outlining how much API’s cost to implement, what cost savings you are likely to see from them; how they reduce content development time; few guidelines on how to deal with syndicated content copyright issues.

Partly, this knowledge gap is because many of the technologies we’re talking about are still quite young. But a lot of the problem is about the communication of technology, the divided worlds that Nick Poole (Collections Trust) speaks about. This was the core of my presentation: ten reasons why MRD is important, from the perspective of a non-geek (links go to relevant slides and examples in the slide deck):

1. Content is still king
2. Re-use is not just good, it’s essential
3. “Wouldn’t it be great if…”: Life is easier when everyone can get at your data
4. Content development is cheaper
5. Things get more visual
6. Take content to users, not users to content (”If you build it, they probably won’t come”)
7. It doesn’t have to be hard
8. You can’t hide your content
9. We really is bigger and better than me
10. Traffic

All this is is a starter for ten. Bigger, better and more informed people than me probably have another hundred reasons why MRD is a good idea. I think this knowledge may be there – we just need to surface and collect it so that more (of the right) people can benefit from these approaches.

I just finished my talk at Mashed Library 2009 – an event for librarians wanting to mash and mix their data. My talk was almost definitely a bit overwhelming, judging by the backchannel, so I thought I’d bang out a quick blog post to try and help those I managed to confuse.

My talk was entitled “Scraping, Scripting and Hacking your way to API-less data”, and intended to give a high-level overview of some of the techniques that can be used to “get at data” on the web when the “nice” options of feeds and API’s aren’t available to you.

The context of the talk was this: almost everything we’re talking about with regard to mashups, visualisations and so on relies on data being available to us. In the cutting edge of Web2 apps, everything has got an API, a feed, a developer community. In the world of museums, libraries and government, this just isn’t the case. Data is usually held on-page as html (xhtml if we’re lucky), and programmatic access is nowhere to be found. If we want to use that data, we need to find other ways to get at it.

My slides are here:

A few people asked that I provide the URLs I mentioned together with a bit of context. Many of the slides above have links to examples, but here’s a simple list for those who’d prefer that:

• http://hoard.it as an example of “intelligent scraping” being used to take on-page content and re-deliver it as “nicer” machine-accessible content
• Yahoo! Pipes being used to scrape segments of this page using the Fetch Page module
• Google Docs being used to scrape this page using the importHTML() function (see Tony Hirst’s excellent blog post for a better example)
• Dapper being used to scrape pages with this shape and display extracted data
• YQL being used to scrape this page and deliver search results into a REST query
• HtTrack for downloading entire websites / sections of websites
• RegEx visual tool
• HTML Tidy, a tool for cleaning up “bad” html, available as both a download and a COM object for use in your scripts
• Using OpenCalais for doing natural language text parsing (example form here)
• Yahoo! Term Extraction – example form here
• Yahoo! Geo
• Freedom of Information (example from Frankie Roberto), OCR (from me) and Amazon Mechanical Turk

Phew. Now I can see why it was slightly overwhelming

Watching Google’s amazing “what is a browser” video (below) it is easy (and I can almost hear the geeks laughing) to assume that these are just stupid people on a bad day. I mean, what the hell is wrong with them? “My browser is Google”? WTF?

The thing is, these aren’t stupid people. They’re just normal people, going about their normal lives doing normal things. And these are the people we’re building websites and interactive experiences for.

The phrase Problem Exists Between Chair And Keyboard (and wow, isn’t it interesting that there are LOADS of phrases on the same page which mean the same thing, and all equally rude..) was invented by developers trying hard to find excuses for the poor implementation they just rolled out.

The thing is, the problem isn’t BCAK, it’s In The Dev Team. Maybe we should invent a new acronym: PEITDT…

Longer term, this is of course more to do with tech literacy, being a digital native, familiarity with the web and so on. Shorter term, until we solve the literacy problem, we need to pay extra-special attention to users. And maybe never, ever say the phrase PEBCAK or any of its permutations again…

It’s been interesting watching the response to whatever 2.0 is as the whatever it was has matured into whatever it is now.

…I should probably rephrase that…

The social web has changed as it crawled its way through those painful teen years of greasy skin, piercings, “you just don’t understand me” and shouting at its sooooo 1.0 parents. And so, too has the institutional assessment of these environments and tools. Once upon a time the development of social tools had our fellow insitutions looking on with horror. After a while it became entirely de rigeur. Round about now, it has become unfashionable to launch anything without some kind of social element.

This is the inevitable Gartner Hype curve in action. We’re right up there at the peak. Everything is exciting, new, spangly. Institutions – not just cultural heritage, but enterprise too – are like kids in a toyshop. Everything we see is exciting. Everything, frankly, also has FUNDING embossed on it in an enormous web2.0 font.

This is inevitable, but irritating. With the rise to the peak of inflated expectations, budgets rise, projects become longer, teams get bigger. In some ways, this should make people like me happy. What we’ve banged on about for so long is at last funded and adopted by institutions. As always, the irritation is more about doing technology for the hell of it rather than looking at how users might really want to interact with our content.

Being at the peak naturally has people considering the trough. Recently, I’ve noticed two cultural heritage commentators taking this kind of angle. Brian Kelly’s recent stop doing, start thinking presentation took our original one (stop thinking, start doing) and turned it around in ways that are probably obvious from the titles. He suggests a more conservative approach to Web 2.0 which looks at risks, balances concerns, considers reliability, accessibility and archiving. Nick Poole does the same – a recent tweet talks about the “luxury of the last ten years” and asks how we should be focussing our efforts from now on.

These are extremely valuable viewpoints. Building our digital strategies on ground that is shifting constantly is a scary thing, and it is absolutely right that we have considered, serious responses to new technologies and the hype. It’s obviously particularly important that we consider this stuff carefully given the current economic climate.

The problem I have is that serious is where people start asking about consequences, are suddenly asked to provide figures on return on investment. Serious is where things slow down and stop being agile. Serious is where Project Managers live. Serious, frankly, isn’t where innovation, fun and excitement happen.

Twitter didn’t grow out of serious. Nor did Facebook. Or Launchball.

Now, I’m convinced that the core proposition of the social web transcends any kind of hype. So ultimately, I think we’ll continue producing online experiences that tend (albeit slowly) towards a viable, fun, user-centric horizon. I also think we’ll come up with the kinds of strategies that Brian and Nick have written about. We need to find ways of safeguarding our approaches, shielding them from the hype as much as is possible. But we need to do this as much (if not more) with big, funded, serious projects (”do it because it addresses user needs, not because you can get funding”) as with the lightweight, agile, rapid ones.

In a sentence: let’s work hard to find coherent and sensible strategies to what we do, but let’s also make sure we continue to innovate, to play, to fail rapidly and then move on. It is here that we’ll likely find true audience engagement.

* An open letter to whoever it may concern at the Department of Culture, Media and Sport *

Dear Sir/Madam

My attention was drawn recently to a Freedom of Information request which was made to you regarding museum web statistics.

The request was made by an ex-colleage and friend of mine, Frankie Roberto, who used the rather lovely WhatDoTheyKnow website to submit it. For those who don’t know, this site lets anyone submit and track FOI requests publically.

As you’ll of course know, the original request went something like this:

Please could you send me the monthly website statistics for all of
the museums which you hold data on, for as far back as the data is
held?

Please also specify which metrics (eg hits, visits, unique
visitors) are used, and which software is used to measure the
statistics (if available).

Frankie submitted this on 15th April 2008.

A fair amount of correspondence seemed to go on between you and Frankie. I won’t repeat the content here. Instead I’d like to jump straight to the last letter on the thread, dated 6th June 2008. Here, you say this:

Following a search of our paper and electronic records, I have established that the information you requested is not held by this Department. We would advise you to seek the information from the website managers for the individual museums in which you have a particular interest.

This caught me slightly by suprise.

For seven years while Head of Web at NMSI I used to (twice yearly, if I recall) – gather and coordinate web statistics for our three national museums (The Science Museum, London; Railway Museum, York and Media Museum, Bradford). And (again, providing my memory hasn’t gone really badly wrong), I seem to remember that it was DCMS who asked for, and received, these stats.

I had a fabulous time working for NMSI, but I can say without hesitation that these six-monthly forays into the depths of log files and Excel spreadsheets were consistently the least pleasant bit of my job. It was important, however: DCMS web stats were, and probably still are, one of the measures by which funding was distributed to national museums. So we knuckled down and got on with it, painful though it was.

It was therefore with a certain amount of concern that I read your letter to Frankie.

Now – I do fully understand that organisations are big and that processes change. I also understand that things get lost. So I’m not going to get hysterical – but I do think it’s important that you maybe go have another look for them. Frankie tells me he doesn’t need the stats any more, but the more I think about it, the more I think it’s important that these are made public and available. At the very least, it’ll make me feel better for those dark Excel days.

I look forward to hearing from you

Regards

Mike Ellis

The concept and importance of museum-based API’s are notions that I’ve written about consistently (boringly, probably) both on this blog and elsewhere on the web. Programmatic and open access to data is – IMO – absolutely key to ensuring the long-term success of online collections.

Many conversations have been going on about how to make API’s happen over the last couple of years, and I think we’re finally seeing these conversations move away from niche groups of enthusiastic developers (eg. Mashed Museum ) into a more mainstream debate which also involves budget holders and strategists. These conversations have been aided by metrics from social media sites like Twitter which indicate that API access figures sometimes outstrip “normal web” browsing by a factor of 10 or more.

On March 4th 2009, Brooklyn Museum announced the launch of their API, the latest in a series of developments around their online collection. Brooklyn occupies a space which generates a fair amount of awe in museum web circles: Shelley Bernstein and team are always several steps in front of the curve – innovating rapidly, encouraging a “just do it” attitude, and most importantly, engaging wholly with a totally committed tribe of users. Many other museum try to do social media. Brooklyn lives social media.

So, as they say – without further ado – here’s Shelley and Paul talking about what they did, how they did it, and why.

Q: First and foremost, could you please introduce yourselves – what your main roles and responsibilities are and how you fit within the museum.

Shelley Bernstein, Chief of Technology. I manage the department that runs the Museum’s helpdesk, Network Administration, Website, gallery technology, and social media.

Paul Beaudoin, Programmer. I push data around on the back-end and build website features and internal tools.

Q: Can you explain in as non-technical language as possible what exactly the Brooklyn API is, and what it lets people do?

SB: It’s basically a way outside programmers can query our Collections data and create their own applications using it.

Q: Why did you decide to build an API? What are the main things you hope to achieve …and what about those age old “social web” problems like authority, value and so-on?

SB: First, practical… in the past we’d been asked to be a part of larger projects where institutions were trying to aggregate data across many collections (like d*hub). At the time, we couldn’t justify allocating the time to provide data sets which would become stale as fast as we could turn over the data. By developing the API, we can create this one thing that will work for many people so it no longer become a project every time we are asked to take part.

Second, community… the developer community is not one we’d worked with before. We’d recently had exposure to the indicommons community at the Flickr Commons and had seen developers like David Wilkinson do some great things with our data there. It’s been a very positive experience and one we wanted to carry forward into our Collection, not just the materials we are posting to The Commons.

Third, community+practical… I think we needed to recognize that ideas about our data can come from anywhere, and encourage outside partnerships. We should recognize that programmers from outside the organization will have skills and ideas that we don’t have internally and encourage everyone to use them with our data if they want to. When they do, we want to make sure we get them the credit they deserve by pointing our visitors to their sites so they get some exposure for their efforts.

Q: How have you built it? (Both from a technical and a project perspective: what platform, backend systems, relationship to collections management / website; also how long has it taken, and how have you run the project?)

PB: The API sits on top of our existing “OpenCollection” code (no relation to namesake at http://www.collectiveaccess.org) which we developed about a year ago. OpenCollection is a set of PHP classes sitting on top of a MySQL database, which contains all of the object data that’s been approved for Web.

All that data originates in our internal collections management systems and digital asset systems. SSIS scripts run nightly to identify approved data and images and push them to our FreeBSD servers for processing. We have several internal workflow tools that also contribute assets like labels, press releases, videos, podcasts, and custom-cropped thumbnails. A series of BASH and PHP scripts merge the data from the various sources and generate new derivatives as required (ImageMagick). Once compiled new collection database dumps and images are pushed out to the Web servers overnight. Everything is scheduled to run automatically so new data and images approved on Monday will be available in the wee hours Tuesday.

The API itself took about four weeks to build and document (documentation may have consumed the better part of that). But that seems like a misleading figure because so much of the API piggy-backs on our existing codebase. OpenCollection itself – and all of the data flow scripts that support it – took many months to build.

Cool diagrams. Every desk should have some.

Q: How did you go about communicating the benefits of an API to internal stakeholders?

SB: Ha, well we used your hoard.it website as an example of what can happen if we don’t! The general discussion centered around how we can work with the community and develop a way people can can do this under our own terms, the alternative being that people are likely to do what they want anyway. We’d rather work with, than against. It also helped us immensely that an API had been released by DigitalNZ , so we had an example out there that we could follow.

Q: It’s obviously early days, but how much interest and take-up have you had? How much are you anticipating?

SB: We are not expecting a ton, but we’ve already seen a lot of creativity flowing which you can check out in our Application Gallery. We already know of a few things brewing that are really exciting. And Luke over at the Powerhouse is working on getting our data into d*hub already, so stay tuned.

Q: Can you give us some indication of the budget – at least ballpark, or as a % compared to your annual operating budget for the website?

SB: There was no budget specifically assigned to this project. We had an opening of time where we thought we could slot in the development and took it. Moving forward, we will make changes to the API and add features as time can be allocated, but it will often need to be secondary to other projects we need to accomplish.

Q: How are you dealing with rights issues?

SB: Anything that is under copyright is being delivered at a very small thumbnail size (100px wide on the longest size) for identification purposes only.

Q: What restrictions do you place on users when accessing, displaying and otherwise using your data?

SB: I’m not even going to attempt to summarize this one. Here’s the Terms of Service – everyone go get a good cup of coffee before settling down with it.

Q: You chose a particular approach (REST) to expose your collections. Could you talk a bit about the technical options you considered before coming to this solution, and why you preferred REST to these others?

PB: Actually it’s been pointed out that our API isn’t perfectly RESTful, so let me say first that, humbly, we consider our API REST-inspired at best. I’ve long been a fan of REST and tend to gravitate to it in principal. But when it comes down to it, development time and ease of use are the top concerns.

At the time the API was spec’ed we decided it was more important to build something that someone could jump right into than something meeting some aesthetic ideal. Of course those aren’t mutually exclusive goals if you have all the dev time in the world, but we don’t. So we thought about our users and looked to the APIs that seemed to be getting the most play (Flickr, DigiNZ, and many Google projects come to mind) and borrowed aspects we thought worked (api keys, mindful use of HTTP verbs, simple query parameters) and left out the things we thought were extraneous or personally inappropriate (complicated session management, multiple script gateways). The result is, I think, a lightweight API with very few rules and pretty accommodating responses. You don’t have to know what an XSD is to jump in.

Q: What advice would you give to other museums / institutions wanting to follow the API path?

SB: You mean other than “do it” <insert grin here>? No, really, if it’s right for the institution and their goals, they should consider it. Look to the DigitalNZ project and read this interview with their team (we did and it inspired us). Try and not stress over making it perfect first time out, just try and see what it yields…then adjust as you go along. Obviously, the more institutions that can open their data in this way, the richer the applications can become.

_______

Many, many thanks to Shelley and Paul for putting in the time to answer my questions. You can follow the development of the Brooklyn Museum collections and API over on their blog, or by following @brooklynmuseum on Twitter. More importantly, go build something cool

I’m just back from Torquay where I’d been asked to speak at the 32nd annual UKSG conference. I first came across UKSG more than a year ago when they asked me to speak at a London workshop they were hosting. Back then, I did a general overview of API’s from a non-technical perspective.

This time around, my presentation was about opening up access to content: the title “If you love your content, set it free?” builds on some previous themes I’ve talked and written about. Presenting on “setting content free” to a room of librarians and publishers is always likely to be difficult. Both groups are – either directly or indirectly – dependent on income from published works. I’m also neither publisher nor librarian, and although I spent some time working for Waterstone’s Online and know bits about the book trade, my knowledge is undoubtedly hopelessly out of date.

Actually, I had two very receptive crowds (thank you for coming if you were there!) and some really interesting debate around the whole notion of value, scarcity and network effects.

Like any sector, publishers and librarians have their own language, their own agendas and their own histories of successes and failures. Also like any sector, they are often challenged to spend time thinking about the bigger picture. Day jobs are about rights and DRM, OPAC and tenure. They aren’t (usually) about user experience, big-picture strategy or considering and comparing approaches from other sectors.

What I wanted to do with the presentation was to look at some of the big challenges which face (commercial) material in the networked world by thinking a bit more holistically about people’s relationship with that content, and the modes of use that they apply to the stuff that they acquire via this networked environment.

The – granted, rather challenging – title of the presentation is actually a question cunningly disguised as a statement. Or maybe it’s a statement cunningly disguised as a question. I lost track. The thing I was trying to do with this questatement (and some people missed this, more fool me for being too subtle) was to say: “Look, here’s how many people are talking about content now: they’re making it free and available; they’re encouraging re-use; they’re providing free and open API’s. They’re understanding that users are fickle, content-hungry and often unfussy about the origin of that content. What, exactly, do we do in an environment like this? What are the strategies that might serve us best? Can we still sell stuff, and if so, how?”

The wider proposition (that content fares rather better when it is freed on the network than when it is tethered and locked down) is a source of fairly passionate debate. I’ve written extensively about Paulo Coehlo’s experiments in freeing his books, about API’s, about “copywrong“, about value, authority and authenticity. The suggestion that if you free it up you will see more cultural capital is starting to be established in museums and galleries. The suggestion that you might, just might, increase your financial capital by opening up is for the most part considered PREPOSTEROUS to publishers. Giving away PDF’s increases book sales? Outrageous. Apart from the only example I’ve actually seen documented, of course, which is Coehlo’s, and that seems to indicate a completely different story.

There are fine – and all the finer the closer you examine them – levels of detail. Yes, an academic market is vastly different from a popular one: you don’t have the scale of the crowd, the articles are used in different ways, the works are generally shorter, the audiences worlds apart. But nonetheless, Clay Shirky’s robust (if deeply depressing) angle on the future – sorry, lack of future – of the newspaper industry needs close examination in any content-rich sector. I don’t think anyone can deny that the core proposition he holds up – that the problems that (newspaper) publishing solves (printing, marketing and distribution) are no longer problems in the networked age. I don’t think that what he’s saying is that we won’t have newspapers in the future, and he’s definitely not saying that we won’t need journalists. What he is saying – and this was the angle I focused on in my slides – is that this change is akin to living through a revolution. And with this revolution needs to come revolutionary responses and understanding that the change is far bigger and more profound than almost anyone can anticipate. The open API is one such response (The Guardian “Open Platform” being an apposite example). Free PDF’s / paid books is another. Music streaming and the killing of DRM is another.

Revolutions are uncomfortable. The wholesale examination of an entire industry is horrifically uncomfortable. Just take a look at the music business and you’ll see a group of deeply unhappy executives sitting around the ashes of a big pile of CD’s as they mourn the good ‘ole times. But over there with music, new business models are also beginning to evolve and emerge from these ashes. Spotify is based on streaming, Last.fm is based on social, Seeqpod is a lightweight wrapper for Google searches, The Pirate Bay ignores everyone else and provides stuff for free.

Which ones are going to work? Which ones will make money? Which ones will work but displace the money-making somewhere else? The simple answer, of course, is that no-one really knows. Some models will thrive, others will fail. Some will pave a new direction for the industry, others we’ll laugh at in five years time.

So where can the answers be found? Predictably for me, I think all sectors (including academic publishing!) need to take a punt and do some lightweight experimentation. I think they need to be trying new models of access based around personalisation, attention data and identity. They need to examine who gets paid, how much and when. They need to be setting stuff free in an environment where they can measure – effectively – the impact of this freedom across a range of returns, from marketing to cultural to financial. If they do this then they’re at least going to have some solid intelligence to use when deciding which models to take ahead. And it may be that this particular industry isn’t as challenged as most people assume, and that the existing models can carry on – lock it down, slap on some DRM, charge for access. It’d be far less uncomfortable if this was the case. But at least that decision would be made with some solid knowledge backing it up.

Open Access is one clear way of forging this debate ahead. But once you get under the apparently simple hood of the OA proposition, it actually turns out that not only are many institutions simply ignoring guidelines to produce OA versions of published works but that the payment models are complicated and based on a historical backdrop which to many seems inherently broken. I’d be interested to hear from someone with way more knowledge than me on the successes and failures or market research done on setting content free in this way.

It was clear to me in talking to a range of people at UKSG – librarians, publishers, content providers – that there are huge swathes of evidence missing – surprising, perhaps, from sectors which pride themselves on accuracy and academic rigour. When I asked “how many people aren’t coming to your site because search engines can’t see your content?” or “what is your e-commerce drop-out rate?” or “how much of your stuff do you estimate is illegally pirated?”, very few had coherent – (or even vague) (or any!) – answers.

More telling, perhaps, is that the informal straw poll question I posed to various people during the conference: “Do you feel that this is a healthy industry?” was almost always answered with a negative response. And when I asked why, the near-consistent reply was: “It’s too complicated; too political; too entangled” or from one person: “the internet has killed us”.

I’m really not as naive as I sometimes appear I know how terribly, terribly hard it is to unpick enormous, political and emotive histories. When I suggest that “we need to start again”, I’m obviously not suggesting that we can wipe the slate clean and redefine the entire value proposition across a multi-billion dollar, multi-faceted industry. But I think – simply – that awareness of the networked environment, a knowledge of how people really use the web today and an open mind that things might need to change in profound ways are very powerful starting points in what will clearly be an ongoing, fraught and fascinating discussion.

There’s been a fair bit of buzz around the launch of the NMOLP (National Museums Online Learning Project) – now apparently renamed as “Creative Spaces” for launch.

I’ve known about this project for a long while – when I was at the Science Museum, very initial discussions were taking place at the V&A about how to search and display collections results from more than one institution. The Science Museum were invited to take part in the project, but in the end didn’t because of resourcing and budgetary issues.

My second touch on the project was from the agency end – the ITT briefly crossed my desk at my current employer, Eduserv. We considered bidding, but in the end decided that it wasn’t a project we could deliver satisfactorily given the particulars of the scope and budget.

Back then – and I think now, although someone from NMOLP will have to confirm – the project was divided into two main sections: a series of “webquests” (online learning experiences, essentially) and a cross-museum collections search. The webquests can be seen here, but I’m not going to consider these in this post because I haven’t had time to spend enough time playing to have an opinion yet.

The Creative Spaces site is at http://bm.nmolp.org/creativespaces/ – at first glance, it’s clean and nicely designed, with a bit of a web2.0 bevel thing going on. It’s certainly visually more pleasing than many museum web projects I’ve seen. The search is quick, and there’s at least a surface appearance of “real people” on the site. I hesitate to use the word “community” for reasons that I’ll highlight in a minute.

There, unfortunately, is where the praise is going to stop. And here’s the three reasons why:

Firstly, this site, much like Europeana (which I’ll get my teeth into in a future post…) utterly fails to grasp what it is about the web that makes people want to engage. I’m astounded that we’re this many years into the social web and haven’t learnt about the basic building blocks for online communities, and are apparently unable to take a step back from our institutional viewpoint and think like a REAL user, not a museum one.

Try – just try – looking at this site with a “normal person” hat on. Now ask yourself: “what do I want to DO here?” or “how can this benefit me?” or “how can I have fun”? Sure, you can create a “notebook” or a “group” (once you’ve logged in, obviously..). The “why” is unclear.

For starters, the functionality is massively underwhelming. Take a look at www.ingenious.org.uk – a NOF digitise project which I worked on maybe 5-6 years ago. Now, I’m not over-proud of this site – it took ages, nearly killed a few people from stress, and the end result could be better, but hey – it has CROSS COLLECTION SEARCH, you can send an ECARD, you can SAVE THINGS TO YOUR LIGHTBOX, you can CREATE A WEB GALLERY. And this was FIVE+ YEARS AGO. Even then, I was underwhelmed by what we managed to do. Now, looking at NMOLP, I’m not underwhelmed – I’m…speechless…

Secondly, there just simply isn’t a reason WHY. Why would I possibly want to create a profile? Where is my incentive? C’mon guys – this is social web 101. Let me quote Wikipedia, when they talk about the Network Effect:

“A more natural strategy is to build a system that has enough value without network effects, at least to early adopters. Then, as the number of users increases, the system becomes even more valuable and is able to attract a wider user base. Joshua Schachter has explained that he built Del.icio.us along these lines – he built an online system where he could keep bookmarks for himself, such that even if no other user joined, it would still be valuable to him“

The other day, I had a Twitter conversation with Giv Parvaneh, the Technical Manager at NMOLP regarding this post, which talks about “monetizing” media. He blogged his response here. Now, we had a minor crossed-wires moment (it’s hard to discuss in 140 chrs) – but my point was not that museums should “monetize” everything (although, I DO think that museums should learn about real business practices, but that’s another post altogether). My point was that users need to feel special to take part. They need to be part of a tribe, a trusted group who can do and say things that they find personally useful. They need experiences with integrity. If you’re not sure what I mean, just spend some time on the Brooklyn Museum collections pages. These guys get it – the “posse“, the “tag game“, the openness. Compare this back to the sterile, shallow experience you have on NMOLP. Now ask yourself – “where would I spend MY time?”. Get it yet?

The second major reason is that, once again, we’re failing to take our content to our users. This is a huge shortfalling of Europeana. People want experiences on their own terms, not on ours. C’mon, let’s not have another collections portal. Spend your social media money adding and updating entries on Wikipedia, or create an object sharing Facebook application. Or just put everything on Flickr. And, please, please, please create an API or at the very least an OpenSearch feed. If the issue is something around copyright – get your arses back to your funders and content providers and sit them down in front of Google images for an hour so they can begin to understand THE INTERNET, before renegotiating terms with them.

The final reason hangs off the search facility. My vested interest here is of course hoard.it – and if you want to hear our rantings about the money spent on big, bad technology projects, then keep an eye out for our Museums and the Web Paper. We aren’t necessarily suggesting that the hoard.it approach should be the technology behind cross-collections searching. But we are suggesting that the approch that NMOLP have taken is expensive, old, clunky and ultimately flawed. When I first saw the technical specification, my response was – “well, why not just spend £20-30k on a Google Search Appliance and simply spider the sites?” – and I haven’t changed. Why re-develop the wheel?

If I was less of a grumpy old man, I’d feel bad about slagging off a new project this hard – I like the people involved, I like the institutions, and I understand the reasons why (museum) projects spiral into directions you probably wouldn’t ever choose. But then I remember that this puppy cost the taxpayer just short of £2 million pounds, and that Europeana will cost €120 million. And then I realise that we have an obligation to keep badgering, nagging, slagging, criticising until someone – finally – gets it right.

At the end of the day, Frankie sums it all up much more succinctly in his email to the MCG list than I do in this post. He simply asks: why?

I was at the Hague during the latter part of last week, doing a keynote at CATCH // Museum 2.0. The organisers had seen me talking at “Kom je ook?” and asked me to go over again.

This talk – “Why the Social Web is here to stay (and what to do about it)” is an expansion on the one I did in December last year at Online Information. That one focused a bit more on the enterprise, wheras this one was specifically pitched at cultural heritage.

The message is much the same: connecting with others is deeply important to people. The social web connects people. Therefore, the social web is deeply important…

Anyway. Here are the slides