ORDS Uptake Project Workshop Report

ORDS Uptake Project Workshop Report

The ORDS Uptake Project workshop was staged on Monday 2nd December 2013 at the Said Business School in Oxford. It gave the opportunity for the Services's initial cohort of early adopters to try out the ORDS interfaces and functionality, and check that the development of the services was heading in the right direction to meet the needs of their research projects. 12 early adopters attended the workshop.

After an introductory presentation updating the group as to the progress of the ORDS development work, each early adopters was given a computer terminal and log-in details to use the system. Two HTML pages indicated in general terms the tasks that the Project Team wanted the researchers to complete during the workshop, but without specific instructions as to how each task should be completed. The idea behind this was this it would force each researcher to 'work it out for themselves' and thus expose elements of the system that were not intuitive and would need to be re-designed. Buttons on the HTML pages could be clicked to provide more detailed instructions to anyone who got stuck. By monitoring remotely which buttons were clicked and how often, the project team could get a sense of which tasks were proving most problematic.

In addition to this, our early adopters were asked to fill in responses to an online questionnaire as they worked through the exercises, so that we could capture and record their thoughts as they worked through the tasks. We used the Bristol Online Surveys tool for this. Each of the exercises was accompanied by the following questions:

  • If you 'got stuck' at any point, please tell us where. Are there any obvious changes we could make to avoid this happening in future?
  • Were there any aspects of the ORDS interface that you found unintuitive - for example, where you couldn't find something, or where the system didn't behave as you expected? Please give as much detail as possible.
  • Were there any aspects of the ORDS interface that you found aesthetically unappealing, or which didn't have the overall look-and-feel you'd expect in a system of this sort? Please give as much detail as possible.
  • If you encountered any bugs, or any actions you took caused the system to crash, please let us know what happened.

Although having several different browser screens open simultaneously caused a bit of confusion at first, the approach seemed to work quite effectively in ensuring the capture of useful feedback.

In between the two sets of hands-on exercises, the online questionnaire asked the early adopters to state how important various aspects of ORDS functionality were for their particular research projects, in order to help us prioritize future functionality development and bug-fixing.

The final page of the online survey asked more general questions about the ORDS, the benefits it would bring, and how (whether) it would change the way our early adopters conduct their research.

Results of the Workshop

We were pleasantly surprised that for most of the exercises, only two or three of the twelve researchers needed to click the buttons for extended help. Whilst there is certainly room to imporove the layout and intuitiveness of the system, and produce more detailed documentation, it seems reasonable to assume that the ORDS service team will not need to personally explain every aspect of functionality to new users.

Aspects of the ORDS which did cause confusion were:
  • Some placing of buttons, and inconsistent labelling
  • Navigation between pages, particularly returning to 'higher level' pages
  • Some respondents felt there was more scrolling than necessary
  • The double sign-in is irritating [we know this and are working on it!]
  • Users need to be able to alter the column widths when viewing data, and the data viewing window should be re-sizable
  • The schema designer doesn't look consistent with the rest of the ORDS
  • It would help if the most important buttons were accessible without having to scroll

Feedback regarding the potential benefits of using the ORDS was very positive, with a number of the early adopters commenting that it would be very useful for sharing and publishing data. Standardising practices and data security were also selling points for some of our testers, and the majority thought that ORDS would ultimatly save them time.

One of the most useful aspects of the feedback was the prioritization of funtionality. Reassuringly the most needful funtions were largely those that we have already implemented or in the process of doing so. The survey confirmed that we need to implement the functionality to deal with images and multimedia as a priority once the 'core' functionality is in place, as well as enabling customizable data-entry forms (which we hadn't anticipated would be such a high requirement. The ability to edit data offline was also considered important, although that will probably something of a re-design and is unlikely to be in place within the next 12 months.

All in all the workshop was extremenly helpful, and we'll be incorporating its findings into our next planning phase in the new year.