Search history & a stateless interface

One of the things I’d like to do for Copac is to re-write the code behind the web based user interface. The current architecture was designed to work with a Z39.50 server and I now consider it to be too complex. This makes it hard to debug when things go wrong and the complexity of it means that things do go wrong.

So, I’d like to move the interface over to a REST based stateless interface that talks dircectly to the database without going through our Z39.50 interface. This should decrease the time to produce a response after a user hits the search button and should be more reliable.

What I wasn’t too sure about, until now, was how we would incorporate Copac’s Search History feature into a stateless, REST based, interface. The answer came to me during the small hours this morning. We can put the searches into the same Atom Publishing Protocol (APP) repository that we plan to use for the Marked List. (The Search History and Marked List would be separate collections within the repository and so wouldn’t be mixed up together.)

The advantages of this are: the user can have an Atom feed of their searches, they can tag and annotate their searches and generally manipulate their search history by deleting and editing entries through APP client software. We might also be able to include searches from other services. I think such a search history would work for any REST based service. So if we can move other Mimas services, such as Zetoc and the Archives Hub over to a REST based interface, then a user could potentially have, in one place, an archive of all the searches they have performed over a number of different services.

Royal College of Surgeons of England catalogue loaded

The library of the Royal College of Surgeons of England holds a collection of national significance. Focussing on surgical and medically related literature, the collection contains materials from the late fifteenth century onwards, including many unique historical materials in a variety of formats. Holdings include tracts and pamphlets, and many paintings, prints, drawings, and photographs, as well as extensive monograph and journal collections. The contemporary collections focus on surgery, dentistry, anatomy, physiology and pathology. The library also holds “an extensive collection of books, tracts and journals concerning the history of surgery, dentistry and medicine.”

The catalogue is being added as part of the Copac Challenge Fund.

An enhanced Marked List

As part of the D2D work we are enhancing the functionality of the “Marked List” feature in Copac. The Marked List allows you to save records from your search session, for downloading or emailing to yourself in a variety of formats. One of the drawbacks to the Marked List is that it is linked to your search session. That means that when you come back to Copac tomorrow, the records you saved today will have gone.

So, one enhancement is to make your List of saved records permanent, so when you come back next week, everything you saved last week is still there. The downside to this is that you will need to login so that we know who you are and which are your records. If you don’t want to login to use Copac, then you will still be able to, you just wont get the facility of a permanent Marked List.

The current plan is to provide an API to the Marked List and it seems most sensible to use the Atom Publishing Protocol (APP). One of the nice side effects of using APP is that you’ll get an Atom feed of the records you’ve saved, plus you’ll be able to manage your collection of records with a suitable APP client outside of the Copac web site. Your Marked List will be private to you, though we will look at adding an option to publish your List to make it public.

The fly in the ointment of all this might be Shibboleth (the UK Academic access management mechanism.) It isn’t clear to me if an Atom feed is going to work in a Shibbolized environment. I hope to have something to test soon and I’ll keep you informed…

Bookmarking Copac records

In a previous post, “Persistent identifiers for Copac records“, I said that we would soon be adding links from our Full record pages to bookmarking sites such as Delicious. Well, we have now added the links to Delicious!

We hope you find this functionality useful. Let us know if you think there are other such sites you think we should be linking to.

Spooky Personalisation (should we be afraid?)

Last Thursday members from the D2D team met up with people from the DPIE 1 and DPIE 2 projects, as well as Paul Walk (representing the IE demonstrator project). The aim was to talk ‘personalisation’ developments for the JISC IE. It’s impossible to cover the entire scope of discussion here (we were at it most of the day). As you might predict, it was a day of heated but engaging debate around a topic that is technically and socially complex. As we think about the strategic future of services and cross-service development, and there are serious questions marks over which direction we’re headed in terms of personalisation (and, of course, if it’s even possible to talk of ‘a’ direction).

The key practical aim of the meeting was to share the personalisation aspects of D2D project work, and also to discuss the recommendations of the two DPIE reports. The D2D work includes some development of personalisation components for Copac, components we are referring to cautiously as ‘lightweight’ for now. One way in which we plan to ‘personalise’ the service for users is by offering a ‘My Local Library’ cross-search, achieved (we hope — we’re very much in early phases here) via a combination of authentication and IP recognition used to identify users’ geographical location, and then a cross-search of local institutional library holdings data via Z39.50 targets.

In addition, by next the middle of next year, Copac users will be able to save marked lists of records and export them into other 2.0 environments via an atom feed (I’ll let Ashley write the more technical post on that development). Further down the line (i.e. beyond the next six months) we are interested in providing tools for users to annotate, bookmark and tag these records, but we also want to make sure that any such developments are not made in isolation and are ‘Copac’-centric — there’s a lot to explore here, obviously.

In and of themselves, these developments are not especially complex — the latter is an example of personalisation via ‘customisation’ (to use JISC definitions) where users explicitly customise content according to their own preferences. What I am especially interested in, however, is how saved lists (‘My Bibliography?’) could be used to potentially support adaptive personalisation (this is what Max Hammond, co-author of the DPIE 2 report wryly referred to at the meeting as ‘spooky’).

Dave Pattern’s experiment with using circulation data to ‘recommend’ items to University of Huddersfield library users is well known, and I hope the first step towards some potentially very interesting UK developments. On this end, we’re interested in knowing if there is anything similar to be gleaned from saved personal lists — ‘users with this item in their saved lists also have…’ (or something along those lines). This terrain is very much untested, and one of the critical issues, of course, is uptake. Amazon’s recommender function is effective due to the sheer number of users (effective *some* of the time, that is — we all have ‘off-base’ Amazon recommendations stories to tell, I admit). And this is just one small example of how adaptive personalisation of a service like Copac (or other JISC IE services) might work — there are also opportunities around capturing attention data, for instance.

The DPIE 2 report urges extreme caution in this regard. It raises some very pointed questions about how JISC and its services should approach adaptive personalisation. Too often, the authors warn, ‘personalisation’ is established as a specific goal, with the assumption that ‘personalisation’ is intrinsically valuable. In this context, change is technology rather than user driven, which is fine for experimental and demonstrator work, but high-risk for established services with a strong likelihood for failure. They question how helpful definitions of Personalisation put forth by JISC are in carrying forward a development agenda (Customisation; Adaptive Personalisation based on Data held elsewhere (APOD); Adaptive Personalisation based on User Activity (APUA). This definition “provides a mix of concepts from data capture to functionality, rather than setting out the logical link between a source of data about a user, a model of that user, and an approach to providing the user with a personal service” (17). Also missing is a robust benefits mapping process — “there is little analysis of the benefits of personalisation, beyond an assertion that it improves the user experience” (20). The report concludes:

Complex developments of “personalisation services” and similar should not be a current
priority for JISC. It seems unlikely that an external personalisation service will be able to
generate a user model which is detailed enough to be of genuine use in personalising
content; user preferences are probably not broadly applicable beyond the specific resource
for which they are set, and user behaviour is difficult to understand without deep
understanding of the resource being used. Attempting to develop user models which are
sufficiently generic to be of use to several services, but sufficiently detailed to facilitate useful
functionality is likely to be a challenging undertaking, with a high risk of failure. (26)

These are somewhat sobering thoughts, especially in a climate of personalisation and 2.0 fervour, but overall the report is useful in considering how to tread the next steps in development activity. Key for us is this issue of the user model — can we (Copac? SUNCAT? JISC?) develop one that is likely to be of use to several services? My hunch right now is ‘no.’ We know very little concrete about researchers’ behaviour and how they might benefit from such tools (interestingly, both DPIE 2 reports focused on benefits for undergraduate students, when most of the services in question are largely used by researchers). About humanities researchers, we know even less (much of the interesting work around online ‘Collaboratories’ centres on the STEM disciplines). Apparently JISC is about to commission some investigative work with researcher-users, and here at Mimas a team is about to undertake some focus group work with humanities researchers to determine how personalisation tools for services like Copac, Intute and the Archives Hub could (or could not) deliver specific benefits to their work. I’m sure this research will prove very useful.

We’re urged to ‘proceed with caution,’ but we proceed nonetheless. At Copac we’re taking a long hard look at what a personalised service might look like, and accepted that some risk-taking is likely forecast for the future. I’m very interested to know other’s opinions on a possible recommender function for Copac — at what level could such a tool prove useful, and when might it possibly be obstructive? Personally, I have used the ‘People who have bought also bought’ feature in Amazon quite extensively as a useful search tool. I am less likely to take up the direct recommendations that Amazon pitches at me through my ‘personalised’ Amazon home page, however. (This comes, in part, from making purchases for a six year old boy. If only I could toggle between ‘mummy’ and ‘professional’ profiles…. now there’s a radical thought).