Expert Usability Evaluation and Learning Audit of an Online Course

You do not have access to learners or you are really short of time. However, you want to find out if your online course is easy to use and learn. What do you do? You can conduct an expert usability evaluation and expert learning audit. Both these evaluation techniques have their roots in usability inspection. Before I proceed further, these techniques have evolved from usability, but have been modified to suit the requirements of the learning field. If you use some other techniques, please do share. We are always eager to learn more effective ways of doing things.

How are these evaluations conducted?
An expert (usability/learning) judges the effectiveness of the course. The expert will go through the course and try everything a learner would if he/she were to take the course. They look for obstacles, ambiguity, functionality, and several other issues that hinder progress. Detailed reports are generated at the end of the evaluation.

What is an expert usability evaluation?
Using this technique, you evaluate the usability of an online course. An expert lists the parameters based on which the evaluation will take place. These could include:

1. Navigation: What is the primary form of navigation? Is this intuitive enough? This would ideally mean that we do not include 'Click Next to proceed' kind of instructions. The learner should intuitively know what the primary navigation is.
   
2. Visual hierarchy: Is the information organization in a logical manner? Eye movement is typically from left to right and from top to bottom. Are all elements positioned keeping this in mind? Will the learner know where the information starts and where it ends?
   
3. Accessibility of information: Are important elements placed upfront? Will the learner be able to access the most important information easily.
   
4. Affordance: Do buttons have the affordance of a click? Will the learner know when a click is required? Will he/she know what is expected on an interactive screen? During learner testing, I have seen learners click images that are not clickable or miss buttons that need to be clicked. This is because the element does not have the affordance of a click. Therefore, it is important to identify such issues.
   
5. Fonts and font sizes: Will the learner be able to read the text easily? Do font colors hinder readability? Are these fonts and font sizes consistent across?
   

What is the difference between a QA and an expert usability evaluation?

• A QA checks whether the online course maps to the signed off storyboard/wireframe. It also checks functionality, consistency, and whether the course has any bugs.
  
• Expert usability evaluation, on the other hand, checks whether the elements in the course are usable. It also takes into account user experience. Does the eLearning application cater to the five principles of usability - learnability, efficiency, memorability, errors, and satisfaction?
  

A QA is more content-centric while an expert usability evaluation is more user-centric. This is the main and the most crucial difference.

What is an expert learning audit?
Using this technique, you can evaluate the learning effectiveness of a course. An expert lists the parameters based on which the evaluation will take place. These could include:

• Learning objectives - content mapping: Are the learning outcomes addressed? Can the content be directly mapped to the learning objectives? Is there more information than is stated in the learning objective?
  
• Learner-content mapping: Is the content specific to the learner profile? Is it relevant? Will it help the learner meet the learning outcomes?
• Learner motivation: Is the course motivating enough for the learner? Why will he/she complete the course? Will they find it interesting? Will he/she be motivated to take an exercise?
• Vizualization: Do the visual elements aid learning? Are they similar in look and feel across the course?
  
• Language: Will the learner understand what is written? Is there any ambiguity?
  

What is the difference between an ID review and learning audit?
By the time the course is developed, several ID reviews have already been done. The learning audit is conducted by a third person who has not been a part of the design phase. Therefore, the course is looked at by a fresh eye and this makes a world of difference. The expert looks at the course without considering the constraints. I believe this is a good thing because several times we compromise on learning because of we are thinking about the constraints. He/she looks to identify the obstacles that will hinder learnability. An expert can help us identify where we have compromised.

Keep the following in mind if you are evaluating:

• Ensure that there are no distractions. This requires a lot of concentration, else you may miss a crucial issue.
  
• Try everything. What is the learner were to click this? What would happen if I go here instead of there?
  
• Use screen grabs to highlight issues. This is helpful as the reader will not have to shuffle between a report (xls, word) and the course.
  
• Include suggestions wherever possible. Provide two or three alternatives if possible. It would be very effective to show a suggestion visually.
  
• Ensure that you mark the repetitive issues as a global comment. But, it is also important to identify all those screens in which the issue is present. This will help save time when the reader is fixing the issues.
  
• If you have set parameters, you could check each screen for each parameter in a logical manner rather than just scan a screen. This way you will not miss anything.
  
• It can become very tedious, tiring, and repetitive. So, be prepared.
  

The reports generated from both the expert usability evaluation and the learning audit is valuable source of feedback. You will be able to identify the things that can be worked on based on the suggestions provided by the expert. Use these techniques to evaluate your online course. Try it once and see how much of a difference it actually makes. But, remember, this is still no match for direct feedback from the learner.