Web Magazine for Information Professionals

Herding Tigers, Part II

Sophie Clarke describes an event designed to share ideas on accessibility, evaluation and the use of learning technology standards.

'Herding Tigers' was held at Oxford University Computing Services on the 17 December 2003, with a theme of Best practice in e-Learning development. This was the second event of its type organised by the University's Learning Technologies Group [1], the previous year's having had a slightly different focus on raising awareness and collaborative working between e-Learning practitioners and academics. This year the day had been devised as an opportunity to discuss some of the practical challenges presented to developers when dealing with the areas of accessibility, applying learning technology standards, and evaluation.


The first topic tackled was accessibility, with an excellent session facilitated by Brian Kelly of UKOLN who had very kindly stepped in at the last minute. Delegates were split up into small groups and discussed their thoughts on a wide range of accessibility-related topics including the end-user experience; education and involvement (particularly of those preparing content); and useful tools and technologies. Each group fed back the fruits of their discussion to the wider group, and it was pleasing to see that there seemed to be something of a consensus as to how accessibility could and should be approached, with a particular emphasis on what was most pertinent to providing e-Learning resources.

The discussion was too lengthy to cover in its entirety here, but here is a summary of some of the main conclusions:

Learning Technology Standards

For the next session participants were thrown in at the deep end with a whistle-stop tour of Interoperability Standards and Tools. There was a wide range of experience across the group and for many people it was their first experience of using standards in a practical capacity. For this reason Howard Noble of the Learning Technologies Group at Oxford had created an open-ended exercise running through the full lifecycle of a content package, with the intention that participants could carry on with it at their leisure later. Participants downloaded a content package from the IntraLibrary Learning Object Management System on the Intrallect Web site [3], and edited the content package in Reload [4], the free editor being developed at the Bolton Institute. The next step was to republish the edited package back to Intralibrary and, time permitting, to advertise it in another repository, MERLOT [5]. The practical session ended with a discussion of next-generation standards-based tools, and in particular the learning design-inspired LAMS (Learning Activity Management System) software.

The hands-on session was immediately followed by lunch, but before getting back to the discussions after the break there was an interesting and technical talk given by David Balch of Technology Assisted Lifelong Learning (TALL), another Oxford-based group. David's presentation covered the workflow that has been gradually refined by TALL [6] over the last few years to specify, produce and deliver online courses using an XML and standards-based approach. Of particular interest was their clever use of MS Word templates, using Word styles to give a familiar content authoring tools for academic authors that can then be converted (almost!) automatically into their in-house XML schema for content.


The evaluation session was run by Martin Oliver of University College London, with a strong component of small-group discussion in much the same vein as the accessibility session from the morning. The focus was on sharing tools and techniques and contemplating some of the political issues arising from evaluation, (an area which most of the audience found particularly useful). Evaluation was considered to be especially hard to get to grips with because strategies need to be varied on a case-by-case basis, and most people looked to using a triangulation of qualitative and quantitative methods to build up an overall picture. Producing at least some numbers at the end of an evaluation was deemed to be a necessary evil as "managers like numbers", although we were not sure what managers themselves would think to this generalisation. Tracking tools and Web logs were a popular choice for providing these numbers (particularly in the age of VLEs). However caution was advised when doing so, given that the numbers are often difficult to analyse meaningfully and do not always show what they seem to; (a hit on a Web page does not tell us if the student understood or even read the material there).

The old problem of actually getting feedback from end-users was something that nearly everybody had had to deal with at some point. A variety of strategies had been tried out ranging from the nice, (bribery in the form of HMV vouchers), to the nasty, (preventing students from accessing course work until a feedback form was received). Not surprisingly there were doubts about the quality of feedback that might be received using some of the more extreme methods! A cheering thought was that in some cases at least, the quantity of feedback was not necessarily all that important, provided that the feedback that was received was useful. Considering motives (political and otherwise) behind carrying out evaluation work was another useful issue raised. It was pointed out that an evaluation that is intended to get together ideas to really improve a Web site might happily result in a list of problems, but if your intention was to get together evidence to show your project in a good light then a list of problems and requests for changes would be not be best information to gather.


The closing discussion was led by Stuart Lee, Head of the local Learning Technologies Group. For a day involving nearly all people on the technology side of resource development, together with a smaller number of academics among them, both parties thought that there was need for a shared vocabulary to link these two groups, and also staff development/educational specialists (a task that JISC is taking on in its Pedagogies for e-learning Programme). A new pedagogic trend was identified, that of creating e-Learning experiences, (rather than the conventional approach of teaching). Students are driving this agenda themselves with their demands for increasingly more sophisticated computer-based resources.

Overall the day was very enjoyable and provided a great deal of food for thought. One of the best points of the day was the opportunity to meet and share with others 'in the same boat'. As one of the participants said: 'There are like-minded souls out there'.


  1. Learning Technologies Group http://www.oucs.ox.ac.uk/ltg/
  2. QA Focus http://www.ukoln.ac.uk/qa-focus/
  3. Intrallect http://www.intrallect.com
  4. Reload http://www.reload.ac.uk
  5. MERLOT http://www.merlot.org/
  6. TALL Development Tools http://www.tall.ox.ac.uk/tallinternet/projects/projects_development-tools.asp

Author Details

Sophie Clarke
Academic Computing Development Team
Oxford University

Email: sophie.clarke@oucs.ox.ac.uk
Web site: http://www.oucs.ox.ac.uk/acdt/

Return to top