paul walk’s weblog

I’ve just been invited in FaceBook to join something called a ‘blog network’. The invitation purported to come from a well-known blogger - someone I’m happy to be associated with. I accepted the invitation, which caused the FaceBook to announce to anyone who cared to notice that I am now a fan of that particular blog.

Err - ’scuse me? I just joined a ‘network’ - I didn’t make any value judgement other than that which can be implied by my joining this network - and I don’t think I implied I was a fan. In this case I’m not too worried by this association as I generally appreciate the blog, but it could be otherwise.

Which brings me to the point (yet another thing I don’t like about FaceBook): It seems to work against any possibility of nuance or sophistication in inter-personal relationships. I can’t have a contact, or a business associate - it’s friendship or nothing.

Tags: blog-network, Facebook

I was invited to my first JISC Innovation Forum which took place over Tuesday and Wednesday of this week, and was held in Keele University. Apart from a smattering of light duties - a couple of meetings, helping to ‘referee’ a session (more later) and taking turns to staff the joint UKOLN / CETIS / OSS Watch / TechWatch stand, I was free to get stuck into the real business of this event which was, for me at least, learning & networking.

The forum has a significant online presence, both in terms of the supporting infrastructure (blogs, transcripts etc.) as well as delegates’ own blog posts etc. - look for stuff tagged with ‘jif08‘.

Some of the highlights of the forum included:

The keynote speech from John Selby, Director (Education and Participation), HEFCE: John gave a really clear outline of some of the issues facing JISC in a changing economic climate. Speaking a little about ’socio-technical’ systems, he offered the view that JISC needs to focus on the nature of such systems, portraying them in terms of the following progression:

Innovation -> Implementation -> Sector change

John suggested that JISC tended to concentrate on communities rather than the sector, that communities could be exclusive and that managing technical and social change together is challenging. We should not take for granted that the buzz within the innovative JISC community is recognised or shared outside this community. He also reminded us that JISC is funded by ‘a tax on the sector’ - I have not heard this description used before.

John offered a stark warning when he described the last decade as a ‘golden era’ in terms of funding and security, and predicted that the next few years would not be so golden.

Finally, John admitted that HEFCE’s strategy is not clear enough, and that HEFCE needs to prioritise and clarify its role in terms of how it deals with the sector and with the Government.

More: transcript and link to presentation slides.

The ‘Identity - Starter for Ten’ session: I was asked to help facilitate this session and I’m very glad to have been involved. It was decided to use a technique known as the ‘gold-fish bowl’ to create a free-flowing ‘debate’, where only two people (out of a room of ~35) could speak at any one time, but the either (or both) of the speakers could be replaced at any moment by any of the other participants. We had a small set of rules to govern proceedings and my role was to ‘referee’ the session - which turned out to be very easy. So easy in fact, that I couldn’t help but join in briefly! The starting discussion was around notions of Identity and the management of this in an institutional context. I imagine that those well-versed in these issues probably didn’t learn anything particularly new, but what transpired was a series of arguments, made by some people with real expertise, which gave a pretty good introduction to this area to those who ‘lurked’ and learned. The feed-back I have received so far has been excellent - here is a short video of a self-described ‘lurker’ (sorry - I don’t know who you are!) explaining how he enjoyed the session (interviewed by Lawrie Phipps of JISC who also proposed the Goldfish Bowl approach in the first place).

More: the gold-fish bowl rules and a transcript.

The closing keynote from Jason DaPonte, Managing Editor, BBC Mobile Platforms: Jason gave an instructive speech about some of the issues the BBC is facing in its quest to ‘deepen the relationship between itself and its users’. He characterised the ‘mobile’ context as having the following characteristics:

• personal
• immediate
• location-aware

An application of this approach might be realised in plans to consider moving from the BBC’s ‘where I live’ paradigm to one of ‘where I am’. In my view, and this is informed by my recent acquisition of a location aware iPhone, this aspect of mobile service delivery is becoming rapidly very interesting.

In another part of his presentation, Jason made reference to a DEMOS report, Making the most of collaboration, which sounds very interesting - I have not yet had time to read it. This report examines the state-of-the-art of ‘public service co-design’ - Jason hinted that the higher education sector did not come out of this too well….

More: transcript and link to presentation slides.

Other good moments included being a part of the winning team at the competition (identifying European countries by map outline, and identifying movie posters) over dinner. My 3G iPhone may have been a contributing factor….

And I did enjoy showing an interactive 3D molecule viewer on the iPhone to Jim Downing and Simon Coles, eminent chemists both, and getting the reaction - ’so what’s the underlying data model?’. To which I could only respond, ‘I have no idea - but look how funky it is….’.

While a few aspects weren’t so good (a bizarre and nearly unusable WIFI service and uncomfortable accommodation) I think the forum was a great success overall - I learned plenty and was able to contribute.

Tags: jif08, JISC

Via my colleague Brian Kelly’s post, I read Catherine O’Brien’s How the Google generation thinks differently on the Times Online site (Brian gets cited offering advice on parenting in a digital age!).

I enjoyed the article, but one sentence in the middle caused me to reminisce about my own childhood, and my approach to ‘doing’ homework:

The experience with which my generation grew up, of absorbing oneself in a single book and allowing its themes to meander into the mind before forming considered judgments, is in danger of being eclipsed by the new, digital world order.

Now I judge myself to be more or less of the same generation as Catherine, but I have a quite different memory of doing homework. As I recall, I spent hours in my bedroom, with a text book or two for sure, but also with Radio Victory playing fairly continuously on my clock-radio. At pre-arranged times I would use my pocket torch to send messages in Morse code to the kid across the other side of the alley-way which ran behind my house. Here’s a sample:

- …. .. … / .– — ..- .-.. -.. / -… . / … — / — ..- -.-. …. / . .- … .. . .-. / .. ..-. / .– . / …. .- -.. / … — — . / -.- .. -. -.. / — ..-. / .–. — -.-. -.- . - / -.. . …- .. -.-. . / ..-. — .-. / … . -. -.. .. -. –. / … …. — .-. - / - . -..- - / — . … … .- –. . … / - — / . .- -.-. …. / — - …. . .-. / .. -. / .–. .-.. .- .. -. / . -. –. .-.. .. … ….

If you feel so inclined, you can translate this using this nifty Morse code translator.

My point is, of course, that continuous partial attention is not a generational phenomenon so much as it is related to age.

And furthermore, while the technology may be different, but thirty years ago I had a remote social network (with two nodes - I didn’t have many friends then, for some reason) which was maintained with a recognised, international standard deployed over a binary protocol using readily-available, commodity hardware.

Tags: morse-code

At a JISC workshop last Thursday I was invited to present some ideas around an architecture to support and exploit repositories in the UK. I gave the presentation the title Repository Architecture #83 .

My intention was to suggest some starting principles and then explore how they held up in the face of real-world issues. Here is the slide where I outlined these principles:

I also asked the question: “do we actually need a new architecture?” - suggesting that there is already a ubiquitous & successful architecture supporting much/most/(all?) of the functionality we want from repositories. Taking a resource oriented approach also seems to offer all kinds of advantages. Applying this approach is certainly not a new idea - others have been here before. However, I suggest that the resource oriented approach and the service oriented approach can be most effective when used to complement each other. I think that there is still be place for the institutional repository as the collection of systems which surround what I call the source repository. I define the ’source repository’ as an (ideally) quite simple system which contains:

• the resources themselves, individually addressed with HTTP URIs
• simple, item-level metadata records
• site-map(s) to aid remote search engines
• public, HTTP interfaces
• feeds to notify remote agents of the deposit of new resources in the repository (RSS and/or Atom)

An ‘institutional’ or ’subject’ or ‘learning object’ repository contains one or more source repositories plus any systems needed to manage it in its particular context. These larger repositories might be very complex: the important point is that the logical component I call the source repository should be as simple as possible in it’s public facing interface: basically a bunch of resources, with an address space. So, a resource is given a Cool URI , and a (probably) simple metadata record is made available, also as a resource with a URI. I suggested that an ORE resource map could be used to relate metadata record to resource - from the point of view of the web or ORE, a metadata record is a resource just like, for example, a PDF of a scholarly paper. Elsewhere more, richer metadata might be created through mechanisms ranging from automatic metadata creation, to further human effort which might be in the nature of traditional cataloguing by trained and motivated individuals, or ‘crowd-sourced’ tagging by untrained but still motivated people.
Complexity is introduced, where necessary, in services developed to manage and exploit resources held in source repositories. Crucially, such activity does not happen unless there is a clear incentive for it, and then it happens close to the point of incentive. As an example, if a particular domain has a strong need to classify papers then someone might go to the trouble of harvesting, aggregating and text-mining the text of these papers with a view to extracting terms to use for classification. Or something similar might be achieved through the application of a team of professional cataloguers using an agreed vocabulary. However it is done, the new metadata thus created could be made available as a web resource where it could be used and combined with other resources as required.
I was asked to illustrate this with a few diagrams which provoked a fair amount of discussion.

The point was made, strongly, that it is subject repositories which have the content, rather than institutional repositories. Regardless of whether this is, or will continue to be true, I think the architectural principles hold up. The business drivers are, I guess, quite different!

I learned a lot from the workshop and had some of these ideas challenged quite robustly. I think they held up but the clarity of presentation could be improved - this is what I will be working on now.

Tags: JISC, ORE, repositories, resource-oriented, ukoln

For some time now I’ve been thinking about what I think of as the ascendency of the opportunistic developer in web application development. The phrase has unfortunate connotations for those who remember the ‘personas’ meme from some years ago when it was revealed that Microsoft had characterised three type of developer for three of its software development products. [1] and [2]. This post is not directly related to these archetypes (the opportunistic developer was called ‘Mort’ in the meme, a name which has become derogatory). Rather, I’m talking abut the developer who, regardless of their ability or their occupation wants to make quick use of something when they discover it, typically on the web.

The opportunistic developer prefers to use someone else’s service/component in the majority of cases. They will create their own software when necessary, and will choose to do so under certain circumstances, but they will accommodate a certain amount of compromise if it means they can get away with using something off-the-shelf. The opportunistic developer is still a developer, as opposed to a power user: they will still write code, just as little as they can get away with.

The proliferation of freely available web-services with simple APIs has created a happy-hunting-ground for the opportunistic developer - a few years ago they were inhibited by a lack of choice of available services to use. In addition to the usual concerns - stability, provenance, price… ease of use is becoming a more important differentiator.

In the JISC Information Environment, the norm has been to develop SOAP interfaces to services, almost by default. There are, no doubt, reasons why this has made sense in the past. However, if there is one thing which became abundantly clear at last week’s IE Demonstrator/CRIG event, it is that institutional repository developers do not want to have to use SOAP interfaces. Aside from the hard-core which is interested in pushing REST as the approach to use in repository-service interactions, the consensus was that the use of SOAP for public service interfaces, rather than being an enabling mechanism, is actually a barrier to adoption.

Whether RESTful or not, services are going to have to start having very good reasons for not offering very simple APIs over HTTP, if they are to attract the opportunistic developer.

Tags: iedmonstrator, JISC IE, opportunistic-developer, ReST, SOAP

I haven’t minted a TLA for ages - I think I might be the the first to come up with PPP for Personal Profile Portability as a convenient handle to wrap around the current flavour of ‘data portability’ being touted by the major ‘walled-garden’ social network sites.

Both MySpace and Facebook have recently launched initiatives to open up a little….but not too much.

MySpace has announced its Data Availability project with some major partner applications. Essentially, this will encourage the user to manage ‘profile’ information on MySpace, with a view to surfacing this information in other, partner applications (initially Yahoo, eBay, Photobucket and Twitter. It will also allow users to share some data such as photos which they have added to the MySpace site. Facebook has a similar initiative called Facebook Connect, initially in partnership with Digg. In both cases, a set of usage policies will be imposed such that the user retains control over what is shared, with the power to revoke the sharing agreement. I’m really encouraged to note that in the case of MySpace’s Data Availability, the mechanism adopted to solve the inter-authentication/authorisation issues between these systems is an implementation of OAuth.

Amit Kapur (MySpace’s Chief Operating Officer) says that Data Availability is:

“…founded first and foremost on allowing users to have comprehensive control over their content and data.”

Dave Morin of Facebook believes that:

“…the next evolution of data portability is [...] about giving users the ability to take their identity and friends with them around the Web, while being able to trust that their information is always up to date and always protected by their privacy settings.”

The extent to which users ‘have control’ over their content and data even while it has been completely locked up within the MySpace and Facebook applications has been argued about extensively. The relationships between these sites, their users, and their users’ data have evolved over the last year or two, as users have become a little more savvy. Pressure from groups such as DataPortability appears to have had an effect, with MySpace also signing up to this recently.

So, it seems as though the walled gardens are opening up, getting ready to participate in the wider web. Or are they?

In a web of distributed social networks, the most likely way in which users might manage their participation would seem (right now) to be through a single entry point. Essentially, if the web of social networks is going to allow ’single-sign-on for the user, and allow a re-use of profile information, and even content across multiple applications, then one model is to give the user a ‘gateway’ service, where they sign-on and manage their ‘account’. Both Facebook and MySpace are going to battle hard to be that gateway service for the masses. Both have accepted that they can no longer remain as a completely walled garden - they must open up, just a little, to avoid being eventually marginalised. But now that they are not totally closed, they may find it difficult to retain control. They may find others are waiting to seize the initiative. Enter Google, and its Friend Connect service.

Friend Connect is different to the previous initiatives from Facebook and MySpace. Google’s new offering is designed to provide a ‘middleware’ services, sitting between the big social networks, and sundry web applications which might want to exploit the new openings in these services. It also utilises components which have been developed with the OpenSocial API. Friend Connect is, I think, a very significant development, because it shows how more distributed social networks might work. It is significant also in a particular detail - notice how Friend Connect can become a social network of sorts simply by integrating existing social networks. Suddenly, the huge headstart enjoyed by Facebook and MySpace doesn’t look so unassailable. This is, presumably, the real reason why Facebook have taken steps to block Friend Connect.

I suggest that because they have been walled gardens for so long, neither Facebook nor MySpace really know how to succeed as middleware. They have always been the destination - never really a component in someone’s workflow. By contrast, Google has always offered services which the user employs en route to a different destination. Google understands this kind of arrangement fundamentally. Expect to see increasingly desperate measures from MySpace and Facebook to retain control while Google quietly grows its Friend Connect service.

Tags: data-availability, Facebook, friend-connect, Google, myspace, oauth, personal-profile-portability, ppp

At the University of Bath, where UKOLN is based, we use an enterprise calendar solution from Oracle. It’s OK, no worse than some others I’ve used, but not great. It does have a client for Mac OSX in its favour…. but I don’t really want to use a dedicated client when I have systems and workflows with dependencies on the Mac’s built in calendar application, iCal. iCal is not particularly great either, but it is integrated into Mac OSX and there are advantages to be had from this. I have a few colleagues who would like to be able to use MS Outlook for similar reasons.

Primarily, I want to be able to keep the calendars on my iPhone and my MacBook in sync. Ideally, I want all of this to be synchronised with the ‘work’ calendar on the Oracle system. Prior to owning the iPhone, I was a long-time Palm user and had configured my Palm to synchronise with the Oracle system via SyncML using the excellent tools from Synthesis. The Palm became the conduit between my MacBook and the Oracle system which while not ideal, was a manageable solution. But I bought the iPhone partly to replace the Palm as a PDA, and having to carry the Palm around to act as a conduit between various calendar systems is not ideal.

The most likely medium-term solution to this will be a promised iPhone SyncML client from Synthesis, which might be available sometime this year once Apple sort out their ‘official’ application supply chain arrangements.

In the meantime, today I decided to see how far I could get using the iCalendar file format for calendar data exchange. The Oracle server does not expose iCalendar files except through SyncML, but it does allow them to be exported and imported via the client. I wondered if the ‘clunky’ process of exporting the work calendar every morning and importing this file into iCal might still be less painful than using a Palm as a syncing device. So I imported the iCalendar file into a new calendar in the iCal application, and experimented with a few things.

When I had finished, I decided to delete local copy of the entire calendar, and it was here that I encountered the dialogue box telling me that doing so would send a notification that this ‘event’ had been deleted. There were two buttons, ‘Cancel’ and ‘Delete and Notify’. I wanted to ‘delete’ but I didn’t want to notify anyone…. I ‘cancelled’. Trying to delete a single ‘event’ in the calendar gave the same message: I clicked ‘Delete and Notify’ to see what would happen. A ‘new email’ window popped up, with my email address pre-filled, and the body of the message explaining that this event had been cancelled.

Now I have real trust in Mac OSX. It’s not always perfect, but it’s rarely bizarre and annoying like some other popular operating system user interfaces I could mention. So I went ahead and deleted the entire local copy of the calendar, pressing the ‘Delete and Notify’ button….. By the time I had reacted sufficiently to get the network cable unplugged (I wasted precious seconds trying to cancel the process and then disabling the WIFI device before remembering I was connected with an ethernet cable) about 70 email messages had been sent to various colleagues. By the time I regained control of my laptop, around 400 messages had been queued up to be sent….

After a little investigation, it transpires that the previous version (Tiger) of Mac OSX’s iCal had three buttons for this dialogue, the other button allowing you to delete without notifying. So this is just a strange oversight perhaps. But it is easy to find forum threads where people have complained about this. It’s enough of a problem that John Maisey has released iCal Reply Checker - a utility which simply addresses this problem. It works fine - although it is a little disconcerting that the buttons haven’t changed their labels, only their effect.

Come on Apple - sort it out!

Tags: Apple, ical, icalendar, iPhone, macosx, palm, syncml, synthesis

Here’s an interesting approach. Bernhard Haslhofer at Media Spaces has developed OAI2LOD Server, a system which harvests metadata with OAI-PMH, processes the records to create a triple store and exposes interfaces to this for linked-data clients, SPARQL clients and web-browsers.

According to the web-page:

The OAI2LOD Server exposes any OAI-PMH compliant metadata repository according to the Linked Data guidelines. This makes things and media objects accessible via HTTP URIs and query able via the SPARQL protocol.

I find myself wondering if there is an application for this software in the institutional repositories space. Leaving the SPARQL aspect aside for a moment, note that this system makes resources available via URLs, having harvested metadata via OAI-PMH. I know from experience that there are all kinds of issues with simply identifying a link to a ‘thing or media object’ in many metadata records harvested from institutional repositories, so how well this works in practice remains to be seen. However, this could provide another approach to getting digital objects buried in repositories exposed as resources in the web-architecture. And while I don’t suppose that OAI2LOD is particularly aimed at institutional repositories, the SPARQL & linked-data interfaces do perhaps offer a route for some suitable repositories to participate in the web of data.

I’m also currently working with large, heterogeneous aggregations of metadata from repositories, so I’m curious to see how this software might fit with that kind of dataset. My guess is that this system will work best with collections which already contain some semantic coherence in the sense that it might suit a subject-based repository rather better than an institutional repository, although the three examples demonstrated on the OAI2LOD site are for national libraries.

So, what’s the real value of this software?. There are some perfectly good alternative systems offering triple stores with similar interfaces. And there is plenty of OAI-PMH harvester software out there. I haven’t seen these two things joined together directly in this way before, which is what has piqued my interest initially. But I assume that the real value must lie in the processing of the metadata records (and other information gleaned as part of the OAI-PMH transaction) into the triple store.

Anyway - it’s an interesting idea coupled with some working code - always a valuable thing in my book!

Tags: linked-data, OAI-PMH, OAI2LOD, SPARQL

Back in February I was asked to give a talk to the JISC Digitisation Programme meeting. I blogged about this shortly beforehand asking for comments and suggestions. The response was fantastic - I received a bunch of great suggestions and incorporated many of them into the presentation. Everyone who commented got a public ‘thankyou’ at the event, and I included all names in the slides I used.

I have finally gotten around to making the slides available (someone who was at the meeting has asked for them so they made some sort of impression with someone!).

Thanks again.

Tags: Digitisation, harvesting, JISC

For some time now I have occasionally advised people involved in repository administration that they should consider registering the Base URL of their OAI-PMH interface (if they have one) with Google as a proxy for a Sitemap. Until recently, Google has supported the use of OAI-PMH Base URLs in its Webmaster Tools which site owners can use to create and register sitemaps in order to give hints about the structure of the website to Google’s web-crawler.

A while ago, I noticed that there was no longer any reference to this particular support in any of the documentation and began to suspect that this was being deprecated. Today, Google announced via their official blog that:

…we’ve found that the information we gain from our support of OAI-PMH is disproportional to the amount of resources required to support it. Fewer than 200 sites are using OAI-PMH for Google Sitemaps at the moment.

In order to move forward with even better coverage of your websites, we have decided to support only the standard XML Sitemap format by May 2008. We are in the process of notifying sites using OAI-PMH to alert them of the change.

Fewer than 200 sites…..

There are a few ways of looking at this. Perhaps ‘open access’ repositories are less concerned with Google rankings than the typical website owner. Perhaps the penetration of OAI-PMH in the world is still below any level that Google could find particularly interesting - certainly they never went to great lengths to advertise this support while it lasted. Clearly, Google have come to the end of a ‘trial period’ for their support for this protocol in their main indexing service.

Can we conclude anything from this? Probably not - surely OAI-PMH can thrive without Google Sitemap support? It certainly plays a fairly significant part in my professional life at present! Or should we view this as a symptom of decline….?

The official Google announcement is here.

Tags: Google, OAI-PMH, Sitemap