Basics
Decide framework
DETERMINE THE GOALS
- What are the high-level goals of the evaluation?
- Who wants it and why?
- The goals influence the approach used for the study.
EXPLORE THE QUESTIONS
Define (sub)goals & (sub)research questions.
CHOOSE EVALUATION APPROACH & METHODS
The evaluation approach influences the methods used, and in turn,
how data is collected, analyzed and presented
IDENTIFY PRACTICAL ISSUES
- Intention: Clarify objectives and hypotheses/claims
- Metrics & measures: What, how and why
- People: Target group & participants
- Activities: Derive activities from use cases
- Context: Social, ethical, physical, etc. aspects
- Technologies: Hardware and software
For example:
- Select users.
- Stay on budget.
- Stay on schedule.
- Find participants.
- Select equipment.
Perform a pilot (trial) study!
DECIDE ABOUT ETHICAL ISSUES
EVALUATE, ANALYZE, INTERPRET AND PRESENT THE DATA
The approach and methods used influence how data
is evaluated, analyzed, interpreted and presented.
IMPACT framework
GOOD EVALUATION
- Establishing convincing arguments for your design solution
- By conducting complementary and regular evaluations at different stages
of your design process using the appropriate evaluation methods
(e.g. summative, formative, expert-based, observational, ...)
- Evaluations should result in insights regarding possible problems and
their causes in order to support refinement of your design specification
- Look at user experience in its full breadth: Effectiveness, efficiency,
satisfaction, learnability, mood, connectedness, ...