Skip to Content

Changes for page Test

Last modified by Sofia Kostakonti on 2022/04/05 14:08
From version Icon 13.1 Icon
edited by Aleksanteri Hämäläinen
on 2022/03/15 11:35
Change comment: There is no comment for this version
To version Icon 60.1 Icon
edited by Veikko Saikkonen
on 2022/04/01 14:48
Change comment: There is no comment for this version

Summary

Details

Icon Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.ahamalainen
1 +XWiki.VSaikkonen
Content
... ... @@ -2,10 +2,13 @@
2 2  
3 3  People with dementia often forget to eat and drink, leading to dehydration, malnutrition and decreased wellbeing in general. Our prototype engages in discourses to remind PwD to have lunch and drink water, using the Nao robot platform. The discourse aims to reming the PwD without causing any anxiety or embarrassment which a traditional "alarm" system could cause, and keep them company throughout these activities.
4 4  
5 -RQ1: "Does the robot cause PwD to eat more regularly?"
5 +RQ1: "Does the robot cause PwD to eat more regularly?"*
6 6  RQ2: "Does the PwD experience less negative emotions, such as agitation, sadness, embarrassment, after the interaction with the 'intelligent' robot?"
7 7  RQ3: "Does the robot remind the PwD of their hunger?"
8 +RQ4: "Does the music make the eating more enjoyable for the PwD?"
8 8  
10 +'*' This research question is difficult due to the practical limitations in designing the experimental setup and as such is left to lesser importance.
11 +
9 9  = Method =
10 10  
11 11  The prototype is evaluated with an in-person experiment with multiple participants. In the experiment, the participants will be asked to pretend to be PwD and act accordingly with/without the prototype.
... ... @@ -42,9 +42,17 @@
42 42  
43 43  For the experiments, we'll be using the NAO robot platform, as well as a laptop for the participants to complete the questionnaires on. The questionnaire will be a combination of questions regarding the emotional state of the participants, their interaction with the robot, and the music included in the interaction. Food will be made available to see and measure how much people will eat.
44 44  
48 +Questionnaires:
49 +Consent Form and Disclaimers
50 +8 questions from the [[EVEA>>https://www.ucm.es/data/cont/docs/39-2013-04-19-EVEA%20-%20Datasheet.pdf]] questionnaire
51 +4 questions from the [[Godspeed>>https://www.bartneck.de/2008/03/11/the-godspeed-questionnaire-series/]] questionnaire
52 +3 food-related questions of our own (5-point Likert scale)
53 +2 music-related questions of our own (5-point Likert scale)
54 +
45 45  == Practicalities ==
46 46  
47 47  Beforehand:
58 +
48 48  * Do a practice round by ourselves
49 49  ** Film this
50 50  * Contact other groups and decide on a time slot
... ... @@ -54,6 +54,7 @@
54 54  * Buy snacks
55 55  
56 56  During:
68 +
57 57  1. Give starting questionnare to fill while people are waiting for the previous participant
58 58  2. Guide the participant to the testing spot
59 59  3. Inform the participant where the snacks are
... ... @@ -63,8 +63,8 @@
63 63  7. Conduct the questionnare for the participant
64 64  8. Give the participant the end-questionnare
65 65  
66 -
67 67  Other practicalities during:
79 +
68 68  * We will use the "Wizard of Oz" method for recognizing agreement and disagreement, to make sure that the whole process does not depend on voice recegnition being good enough
69 69  ** Someone will press eg. "y" and "n" on the keyboard according to the participants answers
70 70  * We will change the order in which the smart and basic versions are for each participant
... ... @@ -71,20 +71,159 @@
71 71  ** this way if someone doesn't show up, we don't get skewed amounts
72 72  
73 73  After:
86 +
74 74  * Analyze results
75 75  
76 76  = Results =
77 77  
91 +The results were gathered from 19 personnel, all of whom interacted first with one version of the robot and then the other. 10 of the participants interacted first with the simple version, nine having their first interaction with the advanced version.
78 78  
93 +== Eating ==
79 79  
80 -= Discussion =
95 +{{html}}
96 +<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/EatingComp.png?rev=1.1" alt="Results on the eating of the test personas" style="display:block;margin-left:auto;margin-right:auto" width=750/>
97 +{{/html}}
81 81  
99 +(% style="text-align:center" %)
100 +Figure: Results on the eating of the test personas during the experiment
82 82  
102 +|=Version|=Did not eat|=Ate|=Would not have eaten without the robot|\\
103 +|Simple|16|3|1
104 +|Advanced|13|6|4
83 83  
106 +Simple robot:
107 +
108 +* 16% ate
109 +* 33% of those would not have eaten without the robot
110 +
111 +Advanced robot:
112 +
113 +* 32% ate
114 +* 67% of those would not have eaten without the robot
115 +
116 +== EVEA (Mood) ==
117 +
118 +{{html}}
119 +<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/MoodChangeDumb.png?rev=1.1" alt="Measured moods and changes for the simple robot" style="display:block;margin-left:auto;margin-right:auto" width=750/>
120 +{{/html}}
121 +
122 +(% style="text-align:center" %)
123 +Figure: Median measured moods for the simple version of the robot
124 +
125 +{{html}}
126 +<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/MoodChangeSmart.png?rev=1.1" alt="Measured moods and changes for the advanced version of the robot" style="display:block;margin-left:auto;margin-right:auto" width=750/>
127 +{{/html}}
128 +
129 +(% style="text-align:center" %)
130 +Figure: Median measured moods for the advanced version of the robot
131 +
132 +(% style="text-align:center" %)
133 +Table: Wilcoxon signed rank test results for the hypothesis that the mood changed during the interaction with the simple robot
134 +
135 +|=Mood|=Happiness|=Anxiety|=Sadness|=Anger
136 +|Statistic|37|5|4|14
137 +|P-value|0.54|0.01|0.01|0.45
138 +
139 +(% style="text-align:center" %)
140 +Table: Wilcoxon signed rank test results for the null hypothesis that the mood changed during the interaction with the advanced robot
141 +
142 +|=Mood|=Happiness|=Anxiety|=Sadness|=Anger
143 +|Statistic|32|11|2|17
144 +|P-value|0.18|0.01|0.01|0.45
145 +
146 +(% style="text-align:center" %)
147 +Table: Wilcoxon signed rank test results for the null hypothesis that the mood decreased during the interaction with the simple robot
148 +
149 +|=Mood|=Anxiety|=Sadness|=Anger
150 +|Statistic|81|53|29
151 +|P-value|0.01|0.00|0.23
152 +
153 +(% style="text-align:center" %)
154 +Table: Wilcoxon signed rank test results for the null hypothesis that the mood decreased during the interaction with the advanced robot
155 +
156 +|=Mood|=Anxiety|=Sadness|=Anger
157 +|Statistic|32|149|52
158 +|P-value|0.00|0.01|0.07
159 +
160 +(% style="text-align:center" %)
161 +Table: Wilcoxon signed rank test results for the null hypothesis that the mood increased during the interaction with the simple robot
162 +
163 +|=Mood|=Happiness
164 +|Statistic|37
165 +|P-value|0.27
166 +
167 +(% style="text-align:center" %)
168 +Table: Wilcoxon signed rank test results for the null hypothesis that the mood increased during the interaction with the advanced robot
169 +
170 +|=Mood|=Happiness
171 +|Statistic|32
172 +|P-value|0.09
173 +
174 +== Music ==
175 +
176 +{{html}}
177 +<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/MusicEnjoyable.png?rev=1.1" alt="Effects of music on the test personnel" style="display:block;margin-left:auto;margin-right:auto" width=1250/>
178 +{{/html}}
179 +
180 +(% style="text-align:center" %)
181 +Figure: Answers of the test personas regarding music
182 +
183 +== Godspeed ==
184 +
185 +{{html}}
186 +<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/friendly-hist.png?rev=1.1" alt="Effects of music on the test personnel" style="display:block;margin-left:auto;margin-right:auto" width=750/>
187 +{{/html}}
188 +
189 +(% style="text-align:center" %)
190 +Figure: Answers to the statement 'I thought the robot was friendly'
191 +
192 +{{html}}
193 +<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/pleasant-hist.png?rev=1.1" alt="Answers to the statement 'I thought the robot was pleasant'." style="display:block;margin-left:auto;margin-right:auto" width=750/>
194 +{{/html}}
195 +
196 +(% style="text-align:center" %)
197 +Figure: Answers to the statement 'I thought the robot was pleasant'
198 +
199 +{{html}}
200 +<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/godspeed-barchart.png?rev=1.1" alt="Godspeed questionnaire median comparison'." style="display:block;margin-left:auto;margin-right:auto" width=750/>
201 +{{/html}}
202 +
203 +(% style="text-align:center" %)
204 +Figure: Median measured Godspeed questionnaire dimensions
205 +
206 +|=Dimension|=Likeability|=Intelligence
207 +|Statistic|36|70
208 +|P-value|0.01|0.17
209 +
210 +(% style="text-align:center" %)
211 +Table: Wilcoxon signed rank test results for the null hypothesis that the advanced robot scored higher in the perceived dimensions
212 +
84 84  = Conclusions =
85 85  
215 +From the results we can see that the more advanced robot
216 +
217 +
218 += Discussion =
219 +
220 +
221 +
86 86  = Appendix =
87 87  
224 +== Experiment introduction for participants ==
225 +
226 +
227 +
228 +Hi, we are <NAME> and <NAME> from the TU Delft Socio-Cognitive Engeering course Group 1, thank you for participating in our prototype evaluation experiment. The experiment is being conducted as a part of the TU Delft course on Socio-Cognitive Engineering and aims to evaluate the prototype designed as a part of the course. The evaluated prototype is based on the Nao robot-platform and is intended to improve the wellbeing of people suffering of dementia.
229 +
230 +Consuming food and/or water can be a consequence of the interaction between you and the robot. Therefore, we would like to ask you if you have any allergies. If you have a form of Diabetes, please let us know before we start the first part of the experiment. You are strongly encouraged to share any other health conditions that can possibly be relevant to take into account when doing an experiment with robots and food with us.
231 +
232 +The link between the stimuli of the Nao-robot and the triggering of epileptic seizures is yet unknown. If you have ever experienced epileptic seizures, please let us know. Then, we could see if any special precautions are needed.
233 +
234 +The experiment will last for approximately 15-20 minutes, and consists of two interaction sections with the Nao robot, as well as questionnaires before, between and after the sections. We kindly ask you to act naturally during the experiment and fill the questionnaires truthfully and intuitively. Remember that we are evaluating the prototypes performance, not yours. You can stop the experiment at any time.
235 +
236 +We will be collecting data of the questionnaires and recording some experiments, do you agree with your experiment being recorded? All data excluding the recordings will be anonymised before analysis and storage. The recordings will not be shared with third parties. After the experiment you have the right to ask for information about the collected data and revoke the right to use it. We kindly ask you not to share any information about the experiment with other participants.
237 +
Do you have any questions?
238 +
88 88  == After research interview ==
89 89  
90 90  Setup:
... ... @@ -105,7 +105,3 @@
105 105   - With the more intelligent version?
106 106   - With the less intelligent version?
107 107  - What did you feel like when the robot suggested you should eat/drink?
108 -
109 -
110 -
111 -
Icon EatingComp.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.VSaikkonen
Size
... ... @@ -1,0 +1,1 @@
1 +83.6 KB
Content Icon
Icon MoodChangeDumb.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.VSaikkonen
Size
... ... @@ -1,0 +1,1 @@
1 +27.5 KB
Content Icon
Icon MoodChangeSmart.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.VSaikkonen
Size
... ... @@ -1,0 +1,1 @@
1 +27.7 KB
Content Icon
Icon MusicEnjoyable.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.VSaikkonen
Size
... ... @@ -1,0 +1,1 @@
1 +145.2 KB
Content Icon
Icon friendly-hist.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.VSaikkonen
Size
... ... @@ -1,0 +1,1 @@
1 +19.2 KB
Content Icon
Icon godspeed-barchart.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.VSaikkonen
Size
... ... @@ -1,0 +1,1 @@
1 +15.9 KB
Content Icon
Icon pleasant-hist.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.VSaikkonen
Size
... ... @@ -1,0 +1,1 @@
1 +19.6 KB
Content Icon