Project home4. Evaluation Methods
Our research topic is:
The effect of goal-based and emotion-based explanations in prompting PwD for physical activity.
We will create two systems, both trying to motivate the PwD to go for a walk in the garden. One will use goal-based explanations and the other emotion-based explanations. Maybe we would also need a control group (no explanation), resulting in three systems.
Independent effect: explanation method
Dependent effect: motivation to go to the garden
Confounding effects: Personal enjoyment of nature, weather,
The between-subject study design fits with the limited time that we have. It also makes sure there's no learning effect like what could occur with a within-subject study. We do have to take into account the potential differences between the groups meaning we cannot take the results as a direct conclusion.
After each evaluation session, the participant will be asked to fill in a questionnaire. There's quite some existing for human-robot interaction. However, they are more about the usability of the system. While we see our system just as a conversational motivator for going outside. So we don't see these questionnaires as fit:
SASSI, SUS (System Usability scale), Godspeed questionnaire, ASA questionnaire, AttrakDiff, SUISQ
All participants of the evaluation will be part of the course. So they will all be familiar with the robot in question. They will all be students at the TU Delft aged 20-25.
Final text
A within-subject designed experiment is when each participant is exposed to more than one experiment under testing. A between-subject design is when participants only do one experiment [1]. With within-subject design, a risk is the so-called 'demand effect', which entails that they might expect the researchers to want certain results, and will then act as such. Another thing that might happen with within-subject design is that participants might experience a learning effect, i.e. learning from the first experiment. [2]
A useful questionnaire is [3], which asks questions regarding the interaction with the robot and its perceives usefulness.
References
[1] Greenwald, A. G. (1976). Within-subjects designs: To use or not to use?. Psychological Bulletin, 83(2), 314.
[2] Seltman, H. J. (2012). Experimental design and analysis (pp. 340)
[3] Heerink, M., Krose, B., Evers, V., & Wielinga, B. (2009, September). Measuring acceptance of an assistive social robot: a suggested toolkit. In RO-MAN 2009-The 18th IEEE International Symposium on Robot and Human Interactive Communication (pp. 528-533). IEEE.