What is the role of ISO standards in the usability of eLearning applications?

Dr. Anders T. Lehr: In addition to the German Industrial Safety Law, the national ordinance governing VDU workplaces, and the German Equal Opportunities for People with Disabilities Act, there are some ISO standards that are especially important for the usability of eLearning applications. For example, section 11 of the DIN EN ISO 9241 standard defines the term "usability" as the extent to which a product can be used by specific users in a specific context in order to reach specific goals effectively, efficiently and satisfactorily.

Standards are based on a consensus for solutions that are usually compiled in regard to a technical problem. They bundle the knowledge and suggestions of experts who specialize in the area related to the standard and are - as indicated above - frequently aligned with governmental regulations.



There are standards that relate to human-machine interaction that apply to the design of eLearning applications and are used for usability evaluation. Among these are DIN EN ISO 9241 (Teil 11, 110, 12), DIN EN ISO 10075, DIN EN ISO 13407, and DIN EN ISO 14915.

What can particular methods - Eye Tracking, formally analytical processes, etc. - say about an eLearning application?

Dr. Anders T. Lehr: According to Machate and Burmester (2003), usability evaluation methods can be categorized as either summative or formative, as well as expert-oriented analytical or user-oriented empirical procedures. Summative evaluation procedures seek to achieve an overall assessment of the usability of an eLearning application that is based fundamentally on quantitative data. In contrast, formative approaches help in the search for detailed improvement possibilities; qualitative data in the form of oral statements, observations, and descriptions are involved. Analytical evaluation procedures as a rule include usability experts in the examination of use problems.

In their work, the experts focus on, e.g. the usability criteria from the DIN EN ISO 9241-110 standard or use heuristics such as "Severity Ratings" based on Nielsen (1993), Schneiderman and Plaisant's "8 Golden Rules for Interface Design" (2005), design", or Norman's "7 Principles of Design". In the user-oriented empirical evaluations, the focus is on the experience of the people who use or are supposed to use the eLearning application.

Several methods can be used to undertake a usability test; the choice depends on the information sought. They include field tests with participant observation, questionnaires, eye tracking, screen recording, heuristics, reaction tests of hand-eye coordination based on Fitts law, and paper-and-pen prototypes. There are also biological investigations that can determine stress levels with the aid of saliva tests. All of them, alone or in combination, can be used to evaluate eLearning applications.

Which of these methods do you consider the pragmatically significant?

Dr. Anders T. Lehr: Based on my experience, the ones I've chosen to describe briefly in the following four cases are very well suited for usability evaluations.

If the eLearning application is still in the planning stage, a prototype for which usability experts can be brought in makes sense. This can be done on paper, on which the design and dialogue of the eLearning application to be produced can be illustrated.

If the eLearning application is still in the development stage, a series of walkthrough tests are a good idea. As occurs during the Agile software development process, developer and users sit together and look at the finished modules and test whether usability has already been achieved.

If the eLearning application has already been produced, learner-centered approaches are much more informative. It is important that the learners not be disturbed while studying. Eye-tracking systems and screen-recording tools, which run "invisibly" in the background, are particularly beneficial here.

If an existing eLearning application needs to be revamped, usability questionnaires are very suitable in helping to evaluate the various aspects of usability. With the results, AB comparisons can be implemented on the physiological and psychological levels.

Which usability factors are especially conducive or impedimental to learning?

Dr. Anders T. Lehr: The term "learning conduciveness" that I use in my usability research is taken from the DIN EN ISO 9241-110 standard. It describes the "ergonomic requirements for office activities involving display devices- that are particularly well suited for the organization of user interfaces and dialogues. Learning conduciveness is related to the features and design of a dialogue that supports and guides the user in learning how to use the interactive system.

In order to test learning conduciveness, the DIN EN ISO 9241-12 standard, which deals with the organization of visual information, is applied. It specifies the factors of clarity, distinctness, compactness, consistence, recognizability, legibility, and intelligibility as test criteria. These qualities can be examined at the physiological level (e.g. visual processing), the psychological level (e.g. cognitive processing and working memory), and the ergonomic level (e.g. through the utilization context as defined in the DIN ISO 9241-11 standard.

If usability problems are discovered, they are first viewed as "learning impediments" in regard to the dialogues and user interface, but this may not apply to the content of the eLearning application. However, these med factors can reduce the effectiveness and efficiency of the eLearning application and have great influence on the learner's the satisfaction in the sense of the learning motivation.

Why has the improvement of eLearning programs' usability progressed so slowly?

Dr. Anders T. Lehr: Today, eLearning programs are no longer used exclusively on computers. The plethora of devices means that eLearning today is increasingly less location confined, but this has the disadvantage that the location where the learning takes place may be degraded in terms of work ergonomics. Noise, bad lighting, displays that are too small or have reflective glare diminish usability. Unfortunately, there are few usability approaches so far that are aimed at these types of learning or use contexts.

Furthermore, some usability concepts involve prohibitive cost and time expenditures. For example, Eye-Tracking systems are sometimes simply outside the acceptable range of small eLearning budgets. Good usability tests require considerable time to prepare an experimental design, invite subjects, and do the testing and evaluation. These are things that can lead to delays in the development of the eLearning program that are hard to estimate.

I don't believe that any off-the-shelf usability methods can be used for an eLearning application. Regardless of whether you want to evaluate a learning-platform, a WBT, a blended-learning program, or a serious game, a certain degree of experience is required in order to be able to make an efficient selection from among the various usability methods. Of course, companies with specialized experience can make up for this lack of experience with usability tests, but it also makes sense to build up in-house usability expertise since this can directly be brought into the optimization of development processes.

I think that usability evaluations of eLearning applications will gain in importance. Some journals, various teams of eLearning researchers, and dedicated seminars with best- practice examples have addressed the topic. From the learners' viewpoint, the attention is a step in the right direction since it will make eLearning applications more effective, more efficient, and more satisfactory.