Skip to Content

Changes for page 4. Evaluation Methods

Last modified by Demi Breen on 2023/04/09 14:54
From version 5.1
edited by Hugo van Dijk
on 2023/04/06 19:01
Change comment: There is no comment for this version
To version 3.2
edited by Mark Neerincx
on 2023/03/23 10:12
Change comment: Added comment

Summary

Details

Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.hjpvandijk
1 +xwiki:XWiki.MarkNeerincx
Content
... ... @@ -19,30 +19,3 @@
19 19  
20 20  
21 21  All participants of the evaluation will be part of the course. So they will all be familiar with the robot in question. They will all be students at the TU Delft aged 20-25.
22 -
23 -
24 -=== Final text ===
25 -
26 -
27 -A within-subject designed experiment is when each participant is exposed to more than one experiment under testing. A between-subject design is when participants only do one experiment [1]. With within-subject design, a risk is the so-called 'demand effect', which entails that they might expect the researchers to want certain results, and will then act as such. Another thing that might happen with within-subject design is that participants might experience a learning effect, i.e. learning from the first experiment. [2]
28 -
29 -
30 -Quite some established questionnaires exist regarding human-robot interaction. However, most are more about the usability of a system where the user has a specific goal. Examples of these questionnaires are SASSI [3], SUS [4], and APA [5].
31 -
32 -
33 -Questionnaires also concerning the robot's perceived likeability and general interaction are GodSpeed [6] and a questionnaire proposed by Herink et al. [7], where the latter is more elaborate.
34 -
35 -
36 -=== References ===
37 -
38 -[1] Greenwald, A. G. (1976). Within-subjects designs: To use or not to use?. //Psychological Bulletin//, //83//(2), 314.
39 -[2] Seltman, H. J. (2012). Experimental design and analysis (pp. 340)
40 -
41 -[3] Hone, K. S., & Graham, R. (2000). Towards a tool for the subjective assessment of speech system interfaces (SASSI). //Natural Language Engineering//, //6//(3-4), 287-303.
42 -
43 -[4] Lewis, J. R. (2018). The system usability scale: past, present, and future. //International Journal of Human–Computer Interaction//, //34//(7), 577-590.
44 -[5] Fitrianie, S., Bruijnes, M., Li, F., Abdulrahman, A., & Brinkman, W. P. (2022, September). The artificial-social-agent questionnaire: establishing the long and short questionnaire versions. In //Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents// (pp. 1-8).
45 -
46 -[6] Bartneck, C. (2023). Godspeed Questionnaire Series: Translations and Usage.
47 -
48 -[7] Heerink, M., Krose, B., Evers, V., & Wielinga, B. (2009, September). Measuring acceptance of an assistive social robot: a suggested toolkit. In RO-MAN 2009-The 18th IEEE International Symposium on Robot and Human Interactive Communication (pp. 528-533). IEEE.
XWiki.XWikiComments[0]
Comment
... ... @@ -1,4 +1,4 @@
1 -In this page, you can provide background information ("Foundation") on evaluation methods, as discussed in the lecture. When relevant for you, available instruments (e.g. questionnaires on emotion, or something like the "affect button", could be briefly summarized). The specific evaluation method is described in "3. Evaluation".
1 +In this page, you can provide background information ("Foundation") on evaluation methods, as discussed in the lecture. When relevant for you, available instruments (e.g. questionnaires on emotion, or something like the "affect button", could be briefly summarized).
2 2  
3 3  Bradley, M. M., & Lang, P. J. (1994). Measuring emotion: the self-assessment manikin and the semantic differential. //Journal of behavior therapy and experimental psychiatry//, //25//(1), 49-59.
4 4