JISC CNI Meeting, Edinburgh

I’ve been at the excellent JISC CNI Meeting in Edinburgh these last two days. Lots of interesting work being described and met some great new people. Some people have asked me to post my slides, so here they are:

Posted in Conference Review | Tagged | Comments Off

OR10 Challenge

In case you missed it, the OR10 Developers Challenge is now live!

Andy McGregor has explained why he thinks you should enter the challenge and, I’m pleased to say, there have been some expressions of intent already. If you do decide to enter, please register your intention on the OR10 Crowdvine forum.

A reminder of the challenge:

Create a functioning repository user-interface, presenting a single metadata record which includes as many automatically created,useful links to related external content as possible.?

We had one comment suggesting that the challenge was limited to dealing with Linked Data – this is certainly not the case – we are interested in linking in its broader sense.

 

Posted in Developers | Tagged , | Comments Off

Draft OR10 Challenge idea

Please note that what follows is a draft.

A few weeks ago I posted some thoughts about a Developer Challenge for OR10, with a plea for ideas for specific challenges. I’m pleased to say that this post got a really good response, with plenty of useful ideas and comments. Thank you to all who responded. I think it fair to say that all of the comments influenced our thinking, but the interest in linking content (most fully expressed by Andy Powell) stood out from several comments, so we have concentrated on trying to create a challenge around the this. While linked data was mentioned often (naturally enough), we wanted to stick to our principle of involving non-developers (or users) as much as possible: this can be difficult when dealing with the more esoteric aspects of linked data. So, after some discussion within the DevCSI team, we have worked up the following challenge:

Create a functioning repository user-interface, presenting a single metadata record which includes as many automatically created, useful links to related external content as possible.

Definitions:

  • “functioning” in this sense means that mockups/screenshots are not sufficient – however a working prototype is OK
  • “related” in this sense means that the external content is related to this particular metadata record in some way.
  • “as many useful links” means that marks will be awarded for useful links, so an interface with fifty meaningless links does not beat one with three genuinely useful links!
  • links must be related to content, not just a system. So, for example, a link to the page at http://www.wikipedia.org is not legitimate, but a link to a specific page in Wikipedia could be. Only one link of each ‘type’ counts: i.e. having four links to URLs which reference ‘topics’ in a given system is fine but will count as one link for the challenge.

Rules:

Entries must come from a team of at least one developer and one person representing users. The entries must be presented, in person, at OR10. If a team is responsible for the entry then not all of the team members need be present at OR10, but at least one team-member must be.

Judging:

The entries will be presented/demonstrated at OR10 in a show and tell session in a room dedicated for this. The show and tell will be open to OR10 delegates to come along and see the presentations as they are being made. These presentations/demonstrations will be video-recorded. There will be an opportunity for those delegates present (the ‘audience’) to ask questions and/or comment on the presentations. There will be a panel of judges who will observe and make notes. The judges will take note of the responses from the audience. Following the show and tell, the judges will privately discuss the entries and draw up a shortlist. The videos of the shortlisted entries will be presented at the conference dinner for the assembled delegates to vote a winner and a runner-up.

The judges will particularly take into account the following:

  • functionality – the links must work and must have been created automatically as part of the repository system
  • usefulness – the usefulness of the links to an end-user of the developed interface must be demonstrated
  • number of links – the number and variety of links will be considered
  • audience reaction – favourable and unfavourable reactions for the audience will be taken into account

General points:

The Challenge will be issued well in advance of the conference, giving people plenty of time to develop an entry. We will make facilities available at OR10 – such as a Developers’ Lounge area, for further work to be done at the conference itself.

We are very interested in any comments people may have about this – we intend to publish the final version of this, and open up the Developer Challenge, at the end of this week.

Posted in Social Software | Tagged , , | 7 Comments

Ideas for the OR10 Developer Challenge?

Update: I have closed comment on this post now. Thank you very much to all who commented and suggested ideas for a challenge. I have now posted a draft Challenge here and would welcome comments on that post. Thanks again!

Through the JISC-funded DevCSI project, UKOLN has been asked to arrange a ‘Developer Challenge’ for the Fifth International Conference on Open Repositories, (OR10) to be held in Madrid in July of this year.

This will be the third consecutive year that the Developer Challenge has been a feature of this conference. Previous challenges have been both competitive and creative.

OR09_dev_challenge.jpg

Photo by Graham Triggs

This year we have been considering doing something slightly different. Previously, a general challenge has been issued, inviting developers to submit prototypes for anything which they feel is relevant and useful to the repository community. But now that the community has a better appreciation of the sort of creativity which developers can bring to these events, we wonder if we might try something a little different.

A general challenge?

We have been thinking about the possibility of the repository community issuing a particular challenge to the developers planning to attend OR10. This could be decided on by the community well in advance of the conference. If we managed to ‘crowd source’ a few ideas, we could organise a simple vote. Something we are trying to do more with the DevCSI project is to get developers together with non-developers from the same ‘domain’ (repositories in this case) – so we are quite interested in pursuing this approach with OR10.

The OR10 organisers have helpfully couched the conference itself in terms of some challenges:

In a world of increasingly dispersed and modularized digital services and content, it remains a grand challenge for the future to cross the borders between diverse poles:

  • the web and the repository
  • knowledge and technology
  • wild and curated content
  • linked and isolated data
  • disciplinary and institutional systems
  • scholars and service providers
  • ad-hoc and long-term access
  • ubiquitous and personalized environments
  • the cloud and the desktop.

Perhaps one or more of these could serve as the inspiration for a more concrete developers challenge?

What this boils down to is finding a challenge in the general area of repositories, recognised as important by the community generally, which could only be met by getting developers to work with non-developers at the conference. For it to be fair, the challenge would need to be non-specific with regard to any particular repository software.

I would welcome some feedback:

  • is this general approach a good idea?
  • do you have any ideas for a challenge?

please feel free to comment her if you have any ideas, or alternatively drop me an email at p.walk@ukoln.ac.uk

Thanks!

Posted in Software Development | Tagged , | 23 Comments

Direction counts!

I took advantage of an offer to upgrade my iPhone 3G to the 3Gs model just before Christmas. I spent some time considering the alternatives, and speculating about what might become available during the next eighteen months of my new contract, but I’ve been more than happy with the 3G so my decision was quite an easy one. The 3Gs offered three main improvements over the 3G:

  • a faster processor
  • a better camera
  • a ‘compass’

At first glance, these improvements seem quite modest. But, as we shall see, they add up to something quite significant.

The feature which attracted me mostly was the better camera. People talked about the paltry 2MP camera on the iPhone 3G but to be honest it wasn’t the resolution that was the problem – 2MP is actually adequate for the sorts of pictures I want to take with a pocket camera. The problem with the camera on the 3G was that it was just a rotten camera. I had a better camera in a Sony Clie PDA some five years ago. The camera on the 3Gs is, indeed, better than that on the previous model. It’s not great, but it is just about usable.

The surprise for me is the impact of the other two features. The faster processor was firmly in my ‘nice to have’ category – a welcome improvement but not especially important to me. Once I tried the new model however, I quickly realised what a difference this has actually made. With the previous model, I had attributed a lack of performance in certain applications to network latency. Essentially, I believed that a few apps were simply a little too advanced for the prevailing networks to serve them well. A good example of this was Evernote, an app which seemed promising but was just too sluggish on the 3G to be very useful to me. On the newest iPhone however, Evernote really flies, and network latency does not often impinge on it’s usability. Having a snappier user interface is always nice – but the 3Gs is so much more responsive as a result of its faster processor (and presumably its increased memory).

I had assumed the compass was, effectively, a gimmick. I could see how it would be occasionally useful to orient myself when using the GoogleMap application for example. But over Christmas I started to play with some of the many astronomy apps available for the iPhone. Several of these take advantage of the iPhone’s built-in GPS receiver and compass, allowing the screen to show the night-sky exactly as it appears to the user based on their location and the direction they are facing. This allowed me, for instance, to identify and point out Jupiter to my actually-quite-impressed-for-once family. Direction counts!

What the iPhone 3Gs offers to its applications is a sense of location and direction. Combined, these properties can afford a powerful new functionality.

During 2009 there was a little buzz about augmented reality, with apps such as Wikitude appearing for Android and iPhone, superimposing text and images over real-time views of the physical environment. While I try to avoid predictions for the new year, I’m confident that augmented reality apps will continue to develop, and will become more interesting, during 2010. All of the hardware ingredients – a fast processor, a decent camera, GPS + compass, are present in the iPhone 3Gs. I’m looking forward to what develops as a result.

Coincidentally, my good friend Peter just alerted me to an application called Star Walk. It doesn’t do anything that several other apps don’t also do, but it does it so beautifully. Like all Apple products, aesthetics count for much with the iPhone. When I fired up Star Walk I had a sudden thought – that the reality had just caught up with the aesthetics of mainstream science fiction. If you have an iPhone 3Gs, I recommend you spend the £1.19 for this application, if only to admire the way it looks.

Happy new year!

IMG_0003.PNG IMG_0004.PNG IMG_0003.PNG

Posted in Cool Stuff | Tagged , , | 2 Comments

An infrastructure service anti-pattern

Last week I outlined an idea, that of the service anti-pattern, as part of a presentation I gave to the Resource Discovery Taskforce (organised by JISC in partnership with RLUK). The idea seemed to really catch the interest of and resonate with several of those members of the taskforce who were present at the meeting. My presentation was in a style which does not translate well to being viewed in a standalone context (e.g. on Slideshare) so I have decided to write it up here. I would very much welcome comments on this. (The presentation will be published on the Resource Discovery Taskforce pages and I will ask for this post to be linked to from there when it does appear).

The following diagram is meant to represent a design ‘pattern’ which I have seen often proposed, and sometimes implemented, in the JISC Information Environment (IE) as well as in the wider higher education (HE) sector in general:

anti-pattern.gif

It is my belief that readers who have been involved with the IE for some time will recognise this, at least in a general sense, if not in specific cases. In this arrangement, an aggregation of data is presented to the end user, through the development of a user-facing application or service. The user-facing service will in almost all cases be a web-interface, somewhat similar to the ‘portal’ concept of old but in a centralised, single, global deployment. Because it is generally accepted to be desirable to make such data available to other services (in keeping with the larger goal of interoperability through open standards), one or more machine interfaces or so-called APIs, giving access to the ‘backend’ of the system, will be offered. What this design pattern aspires to is a service implemented to be both user-facing service and machine-facing infrastructure component.

However, I contend that this is, in fact, what software engineers might call an anti-pattern. An anti-pattern is a design approach which seems plausible and attractive but which has been shown, with practice to be non-optimal or even counter-productive. It’s a pattern because it keeps coming up, which means it’s worth recording and documenting as such. It’s anti, because, in practice, it’s best avoided….

There is much which is implicit in this pattern, so I will attempt to surface what I believe are some hidden assumptions in a new version of this diagram: this is what this design pattern, once implemented, reveals:
anti-pattern-extended.gif

In this second diagram, the orange colouring indicates the parts which actually get built and are supported; the yellow indicates the parts which might get built, but which won’t really be supported as a service – in a sense, this is stuff which is believed to work but actually doesn’t; in the case of the users, the yellow colouring indicates that their demand for this service is believed to exist; those components in the diagram which are neither orange, nor yellow, are the product of little more than speculation. In the end, the investment in creating a user-facing application based on an expectation of future demand which doesn’t materialise is wasted while, at the same time, the investment in providing unused machine interfaces is also wasted.

I believe that this design pattern rests on several assumptions which are actually fallacies, and is, therefore, an anti-pattern.

Fallacy 1: “Build it and they will come”:

While infrastructure services can, indeed should, be developed with future opportunity in mind, it is helpful to have an existing and real demand to satisfy, which the new development addresses. If the service is demonstrably useful to users, and is developed effectively with future opportunity in mind, then there is more chance of the service actually working, and of it being attractive to developers working on future opportunities.

Fallacy 2: Interoperability through additional machine interfaces:

Machine interfaces need as much specification, development, testing an maintenance as user-interfaces. Simply making a machine interface available through the adoption of a platform which has a built-in facility offering some standard interface is not enough. A system which proposes to offer three or four APIs is quite likely not going to support any of them adequately. I have argued before that ‘interoperability is not enough‘: in fact, this arrangement does not often lead to interoperability, let alone actual exploitation of the capability to interoperate.

Fallacy 3: People/organisations who can make good infrastructure are also going to be good at building end-user-facing services (and vice versa):

Effective infrastructure supports services which in turn support end-users. The skills and knowledge required to support service-providers are generally quite different from those needed to deliver good user-facing services.

I call this the infrastructure service anti-pattern because the result comes from conflated requirements to deliver both infrastructure (machine-to-machine interfaces) and compelling user-facing services and applications. The result can be something which satisfies neither requirement. The users, requirements and priorities are often completely different between these two problem spaces. I suggest that the following are some possible reasons for this anti-pattern appearing:

  • funding (naturally) tends to follow services, happy users and, importantly, new features.
  • funders like to see their investment showcased
  • infrastructure is mostly invisible making it hard to ascertain impact from users

Proposals for alternative design patterns

Here is a suggested alternative design-pattern:
better-pattern.gif

In this design pattern, the API is developed before any user-facing application, or at least in parallel. An application is developed to exploit this API based on real users requirements. No service is developed until such requirements can be identified. This means that an API will be developed, and it will be being used in at least one case. Opportunities for third party integration for usage of the service are, ideally, identified beforehand. The API is properly supported from the start, or else the service fails completely. The value proposition being offered for further, opportunistic third-party developments, whether real or imagined, is now real and, crucially, supported.

An interesting alternative to this is the approach of combining the user-facing web pages and the machine-actionable API into one interface, through embedded RDFa for example:
better-pattern2.gif

It remains to be seen how this approach is going to work out over time, but we have seen hints of simpler approaches to combining user and machine interfaces in the past, such as RSS being styled to give a decent human-readable interface, or earlier attempts to do interesting things with XHTML.

I wonder if readers agree that the first diagrams represent an anti-pattern which they recognise. And would the proposed alternatives fare any better?

Posted in Programmable Web, Web Infrastructure | Tagged , , | 23 Comments

IDCC09

There is still time to register for this year’s International Digital Curation Conference in London, although you will need to be quick – I’m told that registration closes on the 25th November.

This year’s conference (the fifth), organised in partnership with the Coalition for Networked Information, has the theme Moving to Multi-Scale Science: Managing Complexity and Diversity. It promises to be an interesting event – see the full programme for more details.

You can keep up with developments in advance of the event itself by reading the Digital Curation Blog (see this particular post for example) and/or following on Twitter etc. using the tag ‘#idcc09′.

Posted in Conference Review | Tagged | Comments Off

Linked, open, semantic?

During an interesting session called the ‘Great Global Graph’ at the CETIS conference this week I formed the opinion that, in the recent rush of enthusiasm for ‘linked data’, three ‘memes’ were being conflated. These next three bullets outline my understanding of how these terms have been used in recent discussions, including the CETIS session:

  • Open data: I see this as something expressed as a philosophy or, in more concrete terms, as a policy, such as that espoused by the UK Government. There are aspects of public ownership in this, but also a philosophical approach based on ‘openness’ and a rejection of the economic idea of value in scarcity of information. I think that specific technology does not come into this really: for example one concrete realisation of this policy in the UK is the Freedom of Information Act under which it is perfectly permissible for a data owner to supply data in any reasonable format and medium. Essentially, I generally take ‘open’ to mean accessible to all, notwithstanding conditions of use.
  • Linked data: This one is trickier, as the term is used in quite a precise way by some proponents, based on the principles of linked data form the W3C. There are others who prefer a looser definition. There have been some well-reshearsed arguments about this, which generally come down to whether or not RDF is a pre-requisite of linked data. I’ve become inclined to use the term in its more precisely defined sense, in recognition of the efforts going on in this space.
  • Semantic Web: This term introduces ‘semantics’ into the mix, by layering on ontologies allowing inferences to be made from the data itself.

It seems that these terms are often used together in the same discussions, and I suspect I could benefit from some separation of concerns in some of these discussions. It seems to me that the following are true:

  1. data can be open, while not being linked
  2. data can be linked, while not being open
  3. data which is both open and linked is increasingly viable
  4. the Semantic Web can only function with data which is both open and linked

Option 1 satisfies, in part at least, the drive to make available to the public data which has been paid for by the public and which might be useful to it. There are those (and I count myself among them) who generally believe that at present, for example, it would be better to quickly make the data open in some useable form than to delay this unduly while it is processed into RDF. However, there is a reasonable case to be made for not polluting information spaces with poorly prepared datasets.

Option 2 is an approach for organisations which want to take a more resource-oriented approach to managing and exploiting internal information assets. In the CETIS session an interesting idea was floated around how such an approach might go a long way to helping organisations address data-quality issues.

Option 3 seems increasingly viable. There is value in the ‘linked’ aspect, regardless of whether or not semantic layers are introduced. This is how the Web works after all, and much of the impetus behind Web 2.0 seems, to me, to have come from a healthy mixture of addressable and accessible information and human-mediated convention (e.g. ‘hackable URLs). Perhaps this is the ‘Great Global Graph’ and it’s just a matter of scale?

I’m very open to comment and argument on any of this. Perhaps I’m worrying unduly about these things being mixed up, but I do sense that this space could benefit from some clarity to match the excitement and endeavour.

Posted in Programmable Web | Tagged , , , | 25 Comments

Not ready to wave goodbye to email

Last week I posted a remark on Twitter:

Can’t help thinking that the idea that Google Wave will replace email rather misses the point….

The first response to this echoed my view on this suggesting that the real nature of Wave is rather harder to explain or understand, and implying that people fall back on a frame of reference with which they are comfortable. It certainly looks as though Google have anticipated this and offered some easily digested marketing messages. However, I also saw responses which suggested that some people still seem to be missing the point. One response insisted that Wave would only be successful if it was ‘integrated’ with email. I must confess that I still don’t understand this – I can’t really imagine what impact an integration between Wave and email would really have.

It seems to me that Wave is an ambitious attempt to exploit the idea that one future for the Web lies in social networked activity clustered around shared artefacts. Such artefacts, often what we still call ‘documents’, have been given the useful label social objects. At the centre of a Wave is a social object, with a series of applied and recorded operational transforms. Wave would therefore seem to be primarily about collaboration, as opposed to email or IM which are primarily concerned with messaging. Another way of looking at this would be to suggest that Wave is ‘object-centric’, as opposed to email which is message oriented with a facility to attach auxiliary objects.

The idea that Wave would replace email seems to be suggesting that we won’t need apples anymore because now we have oranges. This is not to say that Wave might not better fit some use-cases currently served by email – such as the problematic mode of collaborative editing of documents by sharing copies sent as email attachments. But even as we adopt better software for collaboration, there’s not much sign that we’re giving up using email. I don’t know about you, but my email inbox isn’t getting any smaller just because I use Google Docs, IM, Twitter…. Email has been tested quite thoroughly now over a few years, and appears to work quite well for asynchronous messaging!
Wave uses XMPP as its underlying protocol which is both interesting and important, but it is also slightly misleading as it implies an important connection with ‘instant messaging, which I think is illusory and unhelpful.

Wave is possible because the barrier of network latency is gradually being reduced. Real-time collaboration across the global network is now viable for many. Of course Wave is not the only game in town – other interesting approaches (mostly also using a variation on the pubsub paradigm) to the real-time Web, such as pubsubhubbub are being actively developed and experimented with. But Google Wave is important – because it’s Google who are doing it. It will gain a lot of publicity, and will likely play its part in driving a culture change allowing real-time collaboration across the global network to ‘go mainstream’. It should be remembered that Google’s Gmail, the poster-child for Web-based email, is still significantly smaller in terms of users than Yahoo and Hotmail.

Because Wave offers APIs to developers and users out of the box, I think it is going to be difficult to say what shape this new offering from Google will take once a significant number of people are using it. The ability to federate Wave services could be significant in this respect.

Posted in Web Infrastructure | Tagged | 11 Comments