Ever since I started working with museums online, I’ve felt that there is a need for strategic advice to help managers of cultural heritage web presences. There are of course hundreds of thousands of resources if you’ve got technical questions, but not many places where you can ask things like “how should I build my web team and structure my budget?” or “how do I write a strategy or business plan?”.

Facet approached me in July asking whether I’d be interested in authoring something for them, and this seemed like the ideal opportunity to try and answer some of these questions.

My (draft) synposis is as follows:

This book will provide a guide for anyone looking to build or maintain a cultural heritage web presence. It will aim to cater both to those who are single-handedly trying to keep their site running on limited budget and time as well as those who have big teams, large budgets and time to spend.

As well as describing the strategic approaches which are required to develop a successful online presence, the book will contain data and case studies on current practice from large and small cultural heritage institutions. This research will help give the reader an insight into how these institutions manage their websites as well as providing hints and tips on best practice. It will have an accompanying web presence which will provide template downloads and other up-to-date information including links and white papers.

As you’ll see, I have no intention of trying to do this all by myself – over the coming year I’m going to be on the phone to many of you (hide now!) asking how you do what you do, and compiling this into what I hope will be a useful guide.

If you have any ideas about what I should include, or the questions I should be asking – please do get in touch either via this blog or on Twitter at @m1ke_ellis!

As always (just like the argument currently brewing about Free), there are two possible dangers in any debate like this. First, we go into too much detail and lose the view of the house because we’re examining the bricks too closely. Second, we polarise the debate.

I’m good at polarising, being a bear of simple brain – particularly when it comes to copyright. Simply, I don’t think it works in many cases, and I think this particular example holds – on many levels – great reasons as to why not. Cross-country, cross-domain, cross-sector, hidden images, non-hidden images, etc etc. This level of complexity doesn’t hold well with users, and they will abuse, either knowingly or unknowingly.

Having said that, there are clearly two sides to this particular debate, and actually I think both sides are being pretty reasonable. NPG have offered medium sized pictures; Wikimedia has been on the case for some years seeking access to these (arguably) public domain images. The discussion over the detail in this particular case will ramble on; the legal threat will be sorted out of court; everyone will ultimately go away at least semi-happy.

The bigger picture is the more important question, and it is this: why are cultural institutions putting collection (images) online? I ask this as an open question, as un-loaded as it can be (given you probably know where I’m coming from on this).

The possible answers are these (none is mutually exclusive, by the way):

• to sell them / variations of them, such as prints, etc
• to increase exposure to them
• to increase exposure to the holding institution
• to increase ticket sales / physical visits to the holding institution

So with these in mind, I think the important questions in this particular debate are not about the devil detail of cross-country copyright or whether Dcoetzee “should” have done what he did. I think they are:

• does the exposure on Wikimedia increase exposure? (Answer: yes)
• does exposure of hi-res pictures stop people from buying them (Answer: unknown, but possibly not)
• does the exposure of the images improve the standing of the institution (as being a place that “has a great collection”) ? (Answer: yes)
• does the exposure of the images increase click-through to the NPG website (and hence, assuming at least some kind of connection between traffic and physical visits) ? (Answer: unknown – I’m about to submit a FOI request to see if we can find out, but probably yes)
• does the threat of legal action make NPG look good? (Answer: not really)

There’s some great questions here, which I’ve been asking our sector to answer for a while. Where is value in a networked age? How does virtual equate to physical? Does exposure increase or decrease physical sales (go ask Anderson or Gladwell this one…).

Just as a closing thought, I wonder if the NPG will be chasing Yahoo! for this YQL query or Google Images for this one? I suspect not.

I thought I’d use the session to talk about something I’m increasingly interested in – the shifting of the message about machine readable data (think API’s, RSS, OpenSearch, Microformats, LinkedData, etc) from the world of geek to the world of non-geek.

My slides are here:

Here’s where I’m at: I think that MRD (That’s Machine Readable Data – I couldn’t seem to find a better term..) is probably about as important as it gets. It underpins an entire approach to content which is flexible, powerful and open. It embodies notions of freely moving data, it encourages innovation and visualisation. It is also not nearly as hard as it appears – or doesn’t have to be.

In the world of the geek (that’s a world I dip into long enough to see the potential before heading back out here into the sun), the proponents of MRD are many and passionate. Find me a Web2.0 application without an API (or one “on the development road-map”) and I’ll find you a pretty unusual company.

These people don’t need preaching at. They’re there, lined up, building apps for Twitter (to the tune of 10x the traffic which visits twitter.com), developing a huge array of services and visualisations, graphs, maps, inputs and outputs.

The problem isn’t the geeks. The problem is that MRD needs to move beyond the realm of the geek and into the realm of the content owner, the budget holder, the strategist, for these technologies to become truly embedded. We need to have copyright holders and funders lined up at the start of the project, prepared for the fact that our content will be delivered through multiple access routes, across unspecified timespans and to unknown devices. We need our specifications to be focused on re-purposing, not on single-point delivery. We need solution providers delivering software with web API’s built in. We need to be prepared for a world in which no-one visits our websites any more, instead picking, choosing and mixing our content from externally syndicated channels.

In short, we now need the relevant people evangelising about the MRD approach.

Geeks have done this well so far, but now they need help. Try searching on “ROI for API’s” (or any combination thereof) and you’ll find almost nothing – very little evidence outlining how much API’s cost to implement, what cost savings you are likely to see from them; how they reduce content development time; few guidelines on how to deal with syndicated content copyright issues.

Partly, this knowledge gap is because many of the technologies we’re talking about are still quite young. But a lot of the problem is about the communication of technology, the divided worlds that Nick Poole (Collections Trust) speaks about. This was the core of my presentation: ten reasons why MRD is important, from the perspective of a non-geek (links go to relevant slides and examples in the slide deck):

1. Content is still king
2. Re-use is not just good, it’s essential
3. “Wouldn’t it be great if…”: Life is easier when everyone can get at your data
4. Content development is cheaper
5. Things get more visual
6. Take content to users, not users to content (“If you build it, they probably won’t come”)
7. It doesn’t have to be hard
8. You can’t hide your content
9. We really is bigger and better than me
10. Traffic

All this is is a starter for ten. Bigger, better and more informed people than me probably have another hundred reasons why MRD is a good idea. I think this knowledge may be there – we just need to surface and collect it so that more (of the right) people can benefit from these approaches.

Many conversations have been going on about how to make API’s happen over the last couple of years, and I think we’re finally seeing these conversations move away from niche groups of enthusiastic developers (eg. Mashed Museum ) into a more mainstream debate which also involves budget holders and strategists. These conversations have been aided by metrics from social media sites like Twitter which indicate that API access figures sometimes outstrip “normal web” browsing by a factor of 10 or more.

On March 4th 2009, Brooklyn Museum announced the launch of their API, the latest in a series of developments around their online collection. Brooklyn occupies a space which generates a fair amount of awe in museum web circles: Shelley Bernstein and team are always several steps in front of the curve – innovating rapidly, encouraging a “just do it” attitude, and most importantly, engaging wholly with a totally committed tribe of users. Many other museum try to do social media. Brooklyn lives social media.

So, as they say – without further ado – here’s Shelley and Paul talking about what they did, how they did it, and why.

Q: First and foremost, could you please introduce yourselves – what your main roles and responsibilities are and how you fit within the museum.

Shelley Bernstein, Chief of Technology. I manage the department that runs the Museum’s helpdesk, Network Administration, Website, gallery technology, and social media.

Paul Beaudoin, Programmer. I push data around on the back-end and build website features and internal tools.

Q: Can you explain in as non-technical language as possible what exactly the Brooklyn API is, and what it lets people do?

SB: It’s basically a way outside programmers can query our Collections data and create their own applications using it.

Q: Why did you decide to build an API? What are the main things you hope to achieve …and what about those age old “social web” problems like authority, value and so-on?

SB: First, practical… in the past we’d been asked to be a part of larger projects where institutions were trying to aggregate data across many collections (like d*hub). At the time, we couldn’t justify allocating the time to provide data sets which would become stale as fast as we could turn over the data. By developing the API, we can create this one thing that will work for many people so it no longer become a project every time we are asked to take part.

Second, community… the developer community is not one we’d worked with before. We’d recently had exposure to the indicommons community at the Flickr Commons and had seen developers like David Wilkinson do some great things with our data there. It’s been a very positive experience and one we wanted to carry forward into our Collection, not just the materials we are posting to The Commons.

Third, community+practical… I think we needed to recognize that ideas about our data can come from anywhere, and encourage outside partnerships. We should recognize that programmers from outside the organization will have skills and ideas that we don’t have internally and encourage everyone to use them with our data if they want to. When they do, we want to make sure we get them the credit they deserve by pointing our visitors to their sites so they get some exposure for their efforts.

Q: How have you built it? (Both from a technical and a project perspective: what platform, backend systems, relationship to collections management / website; also how long has it taken, and how have you run the project?)

PB: The API sits on top of our existing “OpenCollection” code (no relation to namesake at http://www.collectiveaccess.org) which we developed about a year ago. OpenCollection is a set of PHP classes sitting on top of a MySQL database, which contains all of the object data that’s been approved for Web.

All that data originates in our internal collections management systems and digital asset systems. SSIS scripts run nightly to identify approved data and images and push them to our FreeBSD servers for processing. We have several internal workflow tools that also contribute assets like labels, press releases, videos, podcasts, and custom-cropped thumbnails. A series of BASH and PHP scripts merge the data from the various sources and generate new derivatives as required (ImageMagick). Once compiled new collection database dumps and images are pushed out to the Web servers overnight. Everything is scheduled to run automatically so new data and images approved on Monday will be available in the wee hours Tuesday.

The API itself took about four weeks to build and document (documentation may have consumed the better part of that). But that seems like a misleading figure because so much of the API piggy-backs on our existing codebase. OpenCollection itself – and all of the data flow scripts that support it – took many months to build.

Cool diagrams. Every desk should have some.

Q: How did you go about communicating the benefits of an API to internal stakeholders?

SB: Ha, well we used your hoard.it website as an example of what can happen if we don’t! The general discussion centered around how we can work with the community and develop a way people can can do this under our own terms, the alternative being that people are likely to do what they want anyway. We’d rather work with, than against. It also helped us immensely that an API had been released by DigitalNZ , so we had an example out there that we could follow.

Q: It’s obviously early days, but how much interest and take-up have you had? How much are you anticipating?

SB: We are not expecting a ton, but we’ve already seen a lot of creativity flowing which you can check out in our Application Gallery. We already know of a few things brewing that are really exciting. And Luke over at the Powerhouse is working on getting our data into d*hub already, so stay tuned.

Q: Can you give us some indication of the budget – at least ballpark, or as a % compared to your annual operating budget for the website?

SB: There was no budget specifically assigned to this project. We had an opening of time where we thought we could slot in the development and took it. Moving forward, we will make changes to the API and add features as time can be allocated, but it will often need to be secondary to other projects we need to accomplish.

Q: How are you dealing with rights issues?

SB: Anything that is under copyright is being delivered at a very small thumbnail size (100px wide on the longest size) for identification purposes only.

Q: What restrictions do you place on users when accessing, displaying and otherwise using your data?

SB: I’m not even going to attempt to summarize this one. Here’s the Terms of Service – everyone go get a good cup of coffee before settling down with it.

Q: You chose a particular approach (REST) to expose your collections. Could you talk a bit about the technical options you considered before coming to this solution, and why you preferred REST to these others?

PB: Actually it’s been pointed out that our API isn’t perfectly RESTful, so let me say first that, humbly, we consider our API REST-inspired at best. I’ve long been a fan of REST and tend to gravitate to it in principal. But when it comes down to it, development time and ease of use are the top concerns.

At the time the API was spec’ed we decided it was more important to build something that someone could jump right into than something meeting some aesthetic ideal. Of course those aren’t mutually exclusive goals if you have all the dev time in the world, but we don’t. So we thought about our users and looked to the APIs that seemed to be getting the most play (Flickr, DigiNZ, and many Google projects come to mind) and borrowed aspects we thought worked (api keys, mindful use of HTTP verbs, simple query parameters) and left out the things we thought were extraneous or personally inappropriate (complicated session management, multiple script gateways). The result is, I think, a lightweight API with very few rules and pretty accommodating responses. You don’t have to know what an XSD is to jump in.

Q: What advice would you give to other museums / institutions wanting to follow the API path?

SB: You mean other than “do it” <insert grin here>? No, really, if it’s right for the institution and their goals, they should consider it. Look to the DigitalNZ project and read this interview with their team (we did and it inspired us). Try and not stress over making it perfect first time out, just try and see what it yields…then adjust as you go along. Obviously, the more institutions that can open their data in this way, the richer the applications can become.

_______

Many, many thanks to Shelley and Paul for putting in the time to answer my questions. You can follow the development of the Brooklyn Museum collections and API over on their blog, or by following @brooklynmuseum on Twitter. More importantly, go build something cool

If you haven’t gathered it already you’re either new here or have been seriously thick skinned when I’ve ranted on about why I think IT is crap and what we need to do about that. In a nutshell: IT should help people. It usually hinders them. We should try harder. 

That’s the general thrust of what is actually a very straightforward argument. I seem to spend far too much of my time looking at failed potential and not enough looking at astonishing goodness.

From this you’d probably gather – and you’d be mainly right – that I understand, and have much more time for, “lightweight” than I do “enterprise”. I think you can get closer – much closer – to the true horizon of “good IT” with rapid, lightweight approaches than you ever can with heavy, expensive ones. 

Process hell. Thanks johnbullas / Flickr (click for bigger)

I do, however, also understand the need for control, for backups and safety, for security. The problem I have is that so, so often these processes are over-specified. And the problem with over-specification is that it flies in the face of the way we normally go about our lives.

It’s much more natural for us to turn to the person next door or call a friendly web guy and ask “is there any chance you could just…”. It’s what us social animals are good at – a bit of sharing, communal back-scratching, wheeling and dealing. Look at specifications. The fact is, no one knows how the damn thing is going to work, so how the hell are is anyone going to write it down on week one of the project? Project Management is a similar veil we draw into the process mix and pretend is useful, but find me one person who can accurately forecast their time on a task, and I’ll show you a liar.

This is how we end up tied into a sterile, personality-free wasteland where every item has to be built into a specification or you have to raise a ticket with a “service desk” (generally an oxymoron, IMO) to get, say, Google Analytics code pasted into the footer of your website or redirect a URL.

The problem is: this isn’t how people do things. That’s why it bugs the shit out of every poor sucker who gets tied up in it.

Here’s the question: why can’t we ring the guy up? Why can’t we call in favours, buy the dev a pint, tell him we’ll help him out with some CSS he’s struggling with? This is the way, after all, that most business gets done – someone you know puts in a good word for a friend of yours, he does you a favour, you remember it the next time around. I’m not suggesting that goes too far – there’s a horrific cronyism at the other end of the scale – but I’m just unsure how anyone gets anything creative or effective done when there is no personality in the system. The reason we can’t do this (usually) is that there’s another process (“change control”) underneath. And under that, somewhere, hiding in the dark, is fear. 

Mitigating risk is one thing. Being over-cautious is quite another. It’s also a rapidly shrinking spiral where your systems and processes begin to close down on each other because of fear that “things might go wrong”. Lock it down, prevent the changes, project manage it to death. Kill the project before – gasp – something goes awry.

I think “lightweight” consistently shows that creativity can only come out of flexibility. I also believe that this can – with some creative, human thinking – be translated into “the enterprise”. Maybe I’m naive. Maybe it isn’t turtles all the way down, but Process.

I hope not.

IT is rubbish.

I’ve been thinking about this a lot recently. I went to present at Sage Publishing a couple of weeks ago and had a fascinating time re-calibrating my own personal perspective on “where we all are with IT”. I’d made some assumptions about how much certain technologies had penetrated the “real world” and came away with the realisation that I’m not normal. I understand, use, appreciate SaaS (for example): most users don’t. In the real world, most users haven’t even heard of the idea of editing documents online, let alone actually done it.

There was a second and much more profound moment when I said something about how internal systems don’t often work with each other. Simultaneously, the entire room laughed, nodded, grimaced. This was a moment of understanding, a convergence of a shared experience.

What’s sad is that it is so rare that this shared experience is positive when it comes to IT. IT – as I said in the post – is usually considered a blocker and not an enabler.

I’m so very sure that you’re nodding too that I’m not going to back this up with too much evidence, but just ask yourself when you’ve heard (or said) these:

• “I can’t talk to the people on the service desk because they don’t use a language I understand”
• “I set up a blog to get my content out there because I can’t use the institutional CMS”
• “I would love to collaborate but no one can tell me the best way of doing it”
• “We asked for user-centric but got something hugely complicated”
• “We’ve been waiting for months for a new X system but the procurement process is just sooo slow”
• “Why do I only have 200Mb of email storage at work when my Gmail account has over 6Gb?”
• “How come I have to re-key that information X times?”
• “Why is webmail blocked at work?”
• “We only use 5% of the systems’ capabilities – the rest is just noise”

I want to hear from you if these are unfamiliar. I guarantee my inbox will be empty.

So the question I ask in the title is why?

Lets try and boil this down to a few key principles:

Reason One: IT people don’t think like normal people

IT people – and by this I mean web developers, support staff, academics, systems engineers – are almost always enthusiasts. They often (as per the cliche) live the technology – it’s a way of life and not just a job. These are – really – the people who have optimised PC’s at home that they built themselves. They have their own website running off a home-built CMS. They are asked to “fix my computer” by friends and relatives. They read Stuff and Wired for fun. They relish the moment that the Dabs catalogue comes through the door. They like reading the manual.

This is great – really – after all, this is me, too – but what is important is the realisation that you (Mr IT person) aren’t most people. If you’re an IT person then to think like most people requires extraordinary additional effort.

Reason Two: There is no Problem Between Chair And Computer

We IT types need to learn to accept this dictum: “the majority are right”. The fact that 95% of your users don’t see the “click here” link even though it’s blindingly obvious to you does not mean they are stupid. It means you are wrong. In geek speek, there is no PEBCAC.

Reason Three: Functionality is nothing. Simplicity is everything.

User requirements can usually be boiled down to this single word: Simplicity. We geeks are the only people who are excited about the way in which system X integrates with system Y and delivers XML via a SQL call to database Z. Our users wants to run a search or change their website as quickly, painlessly and simply as possible. It’s a shame (hey, I like a groovy API as much as the next person). Get over it.

Reason Four: IT departments are – historically – there to block

I once heard a (very) senior manager say this: “The IT department isn’t there to allow. Its job is to prevent”. Now, argue the IT team, that’s because it’s a dangerous world out there. We can’t have viruses coming in via webmail (so block webmail). We can’t allow wifi networks (so block those). We can’t have people accessing Facebook (block that). We can’t open port 25 (block it). We can’t allow Firefox (block). Etc.

Here’s the reality: If someone in your organisation wants to bring in a file, they will do. End of. Short of gumming up all CD, USB, IR drives and glueing the network cables to the ethernet port, it’s gonna happen. In a similar way, if your users have 6Gb of file space at home (for free), they simply aren’t going to accept 200 Mb at work (“because it’s expensive to provide storage”).

It’s time for IT departments to realise that they should be providing a service – they should strive to understand their users and their content; they should be reactive, rapid, innovative. In short, they need to work with the people they are there to help.

Reason Five: What We Do Is Complicated. Please Don’t Ask Us To Explain.

We hide behind this one, big-time. IT does not make sense. Even if you’re the most clued up, intelligent, well trained, experienced IT bod in the universe, you simply do not know it all. And that’s because it (sometimes) really is complicated. But we still use this as an excuse to cover up the fact that simple answers are almost always what is required. The user interface (and by this I mean “the interface with the user”) is everything, not just a minor part of the equation. IT departments should be hugely proactive in communicating what they are doing and why. We need to learn to interface with people and not just with the tech.

Where from here?
The more time I spend in this environment, the more I realise that the issues here are often “soft” issues. They aren’t – actually – about the systems themselves. They are about the ways that people interact with each other, the understanding that is given by (and to) IT staff, the communication of what we all do. I’ve been exposed to both sides of the fence: on the one hand being treated as “just a geek” and on the other seeing non-IT staff treated as “that bloody user”. The skills gap is almost entirely about a lack of understanding, on both sides.

If we could only talk a bit more, we’d probably find we got on…

I’ve been pretty interested in the environment which is Google Paris – how it feels, what’s different about it, and why – and I’m blogging about it over on the Eduserv PSG blog.

Head over there if you’re interested…

The Chumby, for those who haven’t come across it before, is a small Linux-based wifi device with screen – er, ok – it’s a computer – which sits on your wireless network. The content it displays is entirely hackable – you can point it at any number of Chumby widgets which sit within the Chumby network. These range from simple clocks to news to just plain weird stuff. The basic idea is that the plain ole’ desktop or web widget is now beginning to show itself in the real world.

I think this is exciting for two main reasons: 1. I reckon that widgets (in general, but right now, web and desktop based ones) are the biggest thing happening to content consumers right now, and 2. Ubiquitous computing / internet of things / the spime – the whole pervasive “internet in real world” thing is going to change the way we use web resources in a big way.

So what else is there? Well, mobiles, obviously – generic computing devices which we all carry with us, everywhere. Offshoots from here include SMS, MMS, QR tagging and mobile web browsing. When commentators like Tom Standage start saying things like

“…mobile phones are the most numerous digital devices on the planet, and truly deserve to be called “personal computers…”,

and major telcos start concentrating on the mobile web (T-Mobile’s “Web n Walk” and Vodafone “The internet is now mobile”), you know that these kind of approaches are leaving the steep bit of the hype curve and entering the mainstream.

Another device in the Chumby space is the Nabaztag. Those lovely types at the Science Museum got me one of these as my leaving present and I’ve been hacking it every since. Marcel (picture on left, looking only mildly like a drug dealer..) connects to the web through my wifi network much like the Chumby and receives emails, weather reports, podcasts. Most usefully, I’ve also discovered that he has much more authority for telling my 3 year old to go to bed than me…

Meanwhile in the museum space, Ross Parry and associates recently presented on the concept of the Live!Label – small screen-based labels for exhibits which can be updated anytime via a wifi network.

The general premise of all of these devices is the same: the real world is where we live and move but the internet can start to be layered on top of that world, Matrix style, rather than separated from it. As wire-free (wifi, 3G, GPRS, EDGE..whatever!) access gets faster and more ubiquitous, this layered internet will begin more and more to play a part in our real, not just virtual, lives.

For those who feel this may be a bit of a fatuous response, a recent post on the Amazon Web Service Blog may provide some more reassurance. They’ve announced that as of October 1st 2007, the Amazon S3 Service Level Agreement is in effect. It guarantees 99.9% monthly uptime, with service credits being paid back against your account for any time below the three nines. It seems likely that EC3 will be next, but this is still a beta service so it’s hardly suprising that they’re not offering it right now…

So there you go. Another reason not to compute in the cloud disappears.

Simon’s presentation focused on a number of things which also feature large in my personal tag cloud. Not ducks (although I like them, too) but:

1. Commoditisation of IT – how the movement from new thing to utility service creates tensions as products move from competitive advantage to the cost of doing business

2. Innovation – how the shift from Today’s Hot Stuff to Tomorrow’s Boredom (or, as Tom Standage puts it, the move towards invisible technology) drives, and is driven by, commoditisation

3. How the “new world” of the API and computing in the cloud becomes a utility service: how in this day and age we should be looking at the cloud for IT services and not building and re-building each time we put an application on the web. It’s a view which I’m pushing as hard as I can whenever I can, and it’s lovely to see such an erudite set of slides on a subject area which isn’t the easiest to explain.

It also turns out that Simon has written about the Internet Of Things, Spimes and a bunch of other stuff which really tickle my interest, but these might have to wait until a later post…

One of the major contrasts with Simons presentation, as I said previously was that he really can present in an amusing and interesting way, which was in sharp contrast to pretty much everyone else at the FOWA conference. His presentation style reminded me very much of Dick Hardt’s now famous Identity 2.0 talk which you should check out if you haven’t already.