4. Evaluation Methods

There are two types of evaluation methods: formative and summative evaluation. Formative evaluation is based on open-ended questions that focus on specific interaction processes, while summative evaluation looks at the overall effect and determines whether the objective has been achieved. Both qualitative and quantitative data can be used to measure these evaluations. Qualitative data is used to explore and identify patterns and themes, while quantitative data is used to describe, explain, and predict outcomes. Combining both types of data is often the best approach to evaluation.

What to measure to assess effects?

• Objective measurements
  • Efficiency: time
  • Effectiveness: performance outcomes (errors, restarts, ...)
• Subjective measurements
  • Satisfaction, pleasure/well-being, mood, excitement, likability
• Validated questionnaires
  • System Usability Scale (SUS)
  • Affect Button 
  • Godspeed questionnaire 

Subjective measurements and questionnaires are best fit to evaluate the project. A set of questions will be formulated and used in a questionnaire that participants can fill in after the experiment. The questions can be found here.

1. Kurniawan, S. (2004). Interaction design: Beyond human-computer interaction by Preece, Sharp, and Rogers (2001), ISBN 0471492787.