Skip to Content

Changes for page 3. Evaluation Methods

Last modified by William OGrady on 2024/04/08 22:22
From version 37.2
edited by Jean-Paul Smit
on 2024/03/26 15:44
Change comment: There is no comment for this version
To version 18.1
edited by Rixt Hellinga
on 2024/03/03 23:25
Change comment: There is no comment for this version

Summary

Details

Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -xwiki:XWiki.jeanpaulsmit
1 +xwiki:XWiki.RixtHellinga
Content
... ... @@ -1,39 +1,29 @@
1 -To ground the design rationale in practice, the prototype of the NAO for the PwD will be evaluated in a formative evaluation with the PwD and their caregivers. The evaluation will investigate the process of how interaction with the NAO can alleviate potential symptoms caused by early-stage dementia in PwD.  Three methods of evaluation will be applied to have reliable and accurate results, as Bethel. et al (2020)[1] suggests.
1 +To ground the design rationale in practice, the prototype of the NAO for the subject will be evaluated in a formative evaluation with the subject and their caregivers. The evaluation will investigate the process of how interaction with the NAO can alleviate potential symptoms caused by early-stage dementia in subjects.  Three methods of evaluation will be applied to have reliable and accurate results, as Bethel. et al (2020)[1] suggests.
2 +Three of these methods could be the Wizard of Oz method, the Thinking out loud method, and the Cooperative Evaluation method. Each of which can be concluded with a self-assessment expressed in questions posed to the subjects. This combination of evaluation methods allows for the results to have an input from the subject as well as input from the experts method.
2 2  
3 -The study will focus on the prototype's potential effects, which are based on the desired value of autonomy as a part of self-direction. Before the study, possible confounding variables need to be examined such as the PwD's attitude towards robots and the **autonomy, relatedness** and **competence**.
4 4  
5 -==== ====
5 +The study will focus on the prototype's potential effects, which are based on the desired value of autonomy as a part of self-direction. Before the study, possible confounding variables need to be examined such as the subject's attitude towards robots and the **autonomy, relatedness** and **security**.
6 6  
7 -== //**Study design variables**// ==
7 +==== Study design claims ====
8 8  
9 -The study will hypothesize on the following variables with regards to the system:
9 +The study will investigate the claims on the following questions:
10 10  
11 +~1. **Autonomy. **Does the design increase the sense of //autonomy //in a subject?
11 11  
12 -1. **Autonomy. **Does the design increase the sense of //autonomy //in a PwD?
13 -1. **Relatedness. **Does the design positively affect the PwD's //affective //state?
14 -11. **Affect**. How do participants feel about using the robot in this state?
15 -11. **Attitude towards Technology**. How do people think about using technology? Are they biased towards the robot before the study?
16 -1. **Competence. **Is the design //competent//; is the design capable enough for the PwD to rely on it?
17 -11. **Memory self-efficacy **(pre-study) How good are participants at remembering information?
18 -11. **Memory recall **(post-study) Can the participant accurately retrieve information through the robot?
13 +2. **Relatedness. **Does the design positively affect the subject's //affective //state? Do subjects //like// the system?
19 19  
15 +NOTE: I feel that maybe the statement above is not really what relatedness means? I think it means feeling connectedness to their relatives
20 20  
21 -For a sample size as small as 20 participants, it is most adequate to apply a within-subjects design (they require fewer participants) [1]. That means there is an approach where every PwD is experiencing all of the conditions examined. A within-subjects design might be prone to confounds such as pre-existing notions in the environment. That is why the attitude towards robots and the pre-study sense of affect and autonomy should be examined and evaluated as such.
17 +3. **Security. **Is the design //dependable//; is the design accessible enough for the subject to rely on it? Does it feel natural?
22 22  
23 -[[image:Socio-Cognitive Engineering - Frame 1(4).jpg]]
24 24  
20 +For a sample size as small as 20 participants, it is most adequate to apply a within-subjects design (they require fewer participants) [1]. That means there is an approach where every subject is experiencing all of the conditions examined. A within-subjects design might be prone to confounds such as pre-existing notions in the environment. That is why the attitude towards robots and the pre-study sense of affect and autonomy should be examined and evaluated as such. Another confounder variable to look into is the study location and environment.
25 25  
26 -=== Surveys ===
22 +[[image:Socio-Cognitive Engineering - Frame 1.jpg]]
27 27  
28 -For autonomy, a validated survey tool is the self-care score as used in the Diabetes research paper of PAL by Neerincx et al.
24 +/
29 29  
30 -Affect can be measured with a button ([[http:~~/~~/ii.tudelft.nl/~~~~joostb/affectbutton_version2_original.html>>url:http://ii.tudelft.nl/~~joostb/affectbutton_version2_original.html]]) that takes less than a minute. It will be used for both pre-experiment and post-experiment evaluation.
31 31  
32 -For Attitude towards robots, we build upon the works of [reference] and create a set of two 1-minute questions.
27 +== References ==
33 33  
34 -A well-suited tool for Memory self-efficacy is the Rivermead behavioural memory test [reference]. We adopt it to fit to the ecological validity and domain of interest in our study.
35 -
36 -
37 -== //**References**// ==
38 -
39 -[1] Bethel, C.L., Henkel, Z., Baugus, K. (2020). Conducting Studies in Human-Robot Interaction. In: Jost, C., //et al.// Human-Robot Interaction. Springer Series on Bio- and Neurosystems, vol 12. Springer, Cham. https:~/~/doi.org/10.1007/978-3-030-42307-0_4
29 +(1) Bethel, C.L., Henkel, Z., Baugus, K. (2020). Conducting Studies in Human-Robot Interaction. In: Jost, C., //et al.// Human-Robot Interaction. Springer Series on Bio- and Neurosystems, vol 12. Springer, Cham. https:~/~/doi.org/10.1007/978-3-030-42307-0_4
Socio-Cognitive Engineering - Frame 1(2).jpg
Author
... ... @@ -1,1 +1,0 @@
1 -xwiki:XWiki.jeanpaulsmit
Size
... ... @@ -1,1 +1,0 @@
1 -2.1 MB
Content
Socio-Cognitive Engineering - Frame 1(4).jpg
Author
... ... @@ -1,1 +1,0 @@
1 -xwiki:XWiki.jeanpaulsmit
Size
... ... @@ -1,1 +1,0 @@
1 -2.2 MB
Content
XWiki.XWikiComments[0]
Author
... ... @@ -1,1 +1,0 @@
1 -xwiki:XWiki.MarkNeerincx
Comment
... ... @@ -1,1 +1,0 @@
1 -It would be good to consider, some additional (specific) measures related to memory, like recall and (memory) self-efficacy.
Date
... ... @@ -1,1 +1,0 @@
1 -2024-03-18 16:01:02.566
XWiki.XWikiComments[1]
Author
... ... @@ -1,1 +1,0 @@
1 -xwiki:XWiki.jeanpaulsmit
Comment
... ... @@ -1,1 +1,0 @@
1 -Thank you for this valuable suggestion, we adjusted our study design based on it already. Now, would you say that 4-5 dependent variables is too much to evaluate in our study or will it suffice within the time? For example Affect can be measured with a button ([[http:~~/~~/ii.tudelft.nl/~~~~joostb/affectbutton_version2_original.html>>http://ii.tudelft.nl/~~joostb/affectbutton_version2_original.html]]) that takes less than a minute, but the other variables will add up some time.
Date
... ... @@ -1,1 +1,0 @@
1 -2024-03-21 17:03:29.591
Reply To
... ... @@ -1,1 +1,0 @@
1 -0