Changes for page Test

Last modified by Sofia Kostakonti on 2022/04/05 14:08

From 39.1 to 40.1 From 86.1 to 87.1

From version

40.1

edited by Veikko Saikkonen
on 2022/03/31 17:33

Change comment: There is no comment for this version

To version

86.1

edited by Aleksanteri Hämäläinen
on 2022/04/03 17:59

Change comment: There is no comment for this version

Raw
Rendered

Summary

Page properties (2 modified, 0 added, 0 removed)
Attachments (0 modified, 1 added, 0 removed)
- godspeed-barchart.png

Details

Page properties

Author

@@ -1,1 +1,1 @@
--XWiki.VSaikkonen
++XWiki.ahamalainen

Content

@@ -2,41 +2,53 @@
  People with dementia often forget to eat and drink, leading to dehydration, malnutrition and decreased wellbeing in general. Our prototype engages in discourses to remind PwD to have lunch and drink water, using the Nao robot platform. The discourse aims to reming the PwD without causing any anxiety or embarrassment which a traditional "alarm" system could cause, and keep them company throughout these activities.
--RQ1: "Does the robot cause PwD to eat more regularly?"
--RQ2: "Does the PwD experience less negative emotions, such as agitation, sadness, embarrassment, after the interaction with the 'intelligent' robot?"
--RQ3: "Does the robot remind the PwD of their hunger?"
--RQ4: "Does the music make the eating more enjoyable for the PwD?"
++The four research questions studied in this evaluation are:
++{{html}}
++
++<ol>
++<li>Does the robot remind the PwD of their hunger?</li>
++<li>Does the music make the eating more enjoyable for the PwD?</li>
++<li>Does the PwD experience less negative emotions, such as agitation, sadness, embarrassment, after the interaction with the 'intelligent' robot?</li>
++<li>* Does the robot cause PwD to eat more regularly?</li>
++</ol>
++
++* This research question is difficult due to the practical limitations in designing the experimental setup and as such is left to lesser importance.
++
++{{/html}}
++
  = Method =
--The prototype is evaluated with an in-person experiment with multiple participants. In the experiment, the participants will be asked to pretend to be PwD and act accordingly with/without the prototype.
++The prototype is evaluated with an in-person experiment with multiple participants.
  == Participants ==
--As there are practical difficulties with conducting the experiment with actual people with dementia due to both time constraints and COVID, our participants' group will consist of peers from other groups and friends, who will act as if they are older people with dementia. We plan to gather around 20 people for our experiments.
++As there are practical difficulties with conducting the experiment with actual people with dementia due to both time constraints and COVID, our participants' group consists of peers from other groups and friends. In total we had 19 people take part in our experiment.
  == Experimental design ==
--We will be using a within-subject design. In the experiment all of the participants will interact with both versions of the robot, with half of the participants interacting with the version 1 first and then version 2, and the other half in reverse order, to counter-balance the carryover effects. Snacks will be made available for the participants, in case they're prompted and they're hungry. The participants will be unaware of the possibility of eating snacks, to prevent disturbing the interaction with the robot. Otherwise the subjects could be primed for eating, which would bias the results and hide the effect of the robotic interaction.
++For the experiment we used a within-subject design. All of the participants interacted with both versions of the robot, with half of the participants interacting with the version 1 first and then version 2, and the other half in reverse order. This was done to counter-balance the carryover effects. Snacks were made available for the participants, in case they were prompted and they ewre hungry. The participants were unaware of the possibility of eating snacks, to prevent disturbing the interaction with the robot. Otherwise the subjects would have been primed for eating, which would have biased the results and hide the effect of the robotic interaction.
  == Tasks ==
--The participant will have to interact with the robot, which is programmed to engage in a lunch discourse. Two versions will be implemented: the first version will ask basic questions about mealtime, mostly acting as a reminder for the PwD to have lunch (alarm clock). The second will be our original implementation of it with the more sophisticated discourse and music.
++The participant interacted with the robot, which was programmed to engage in a lunch discourse. Two versions were implemented: the first version asks basic questions about mealtime, mostly acting as a reminder for the PwD to have lunch (basically an alarm clock). The second is our original implementation of it with the more sophisticated discourse and music.
  == Measures ==
--We plan on measuring the effectiveness of the discourse, both physically and emotionally. Our quantitative measure is whether the person ate the lunch they were supposed to have eaten, and the qualitative measure is the emotions that the PwD experienced before, during, and after the interaction. The qualitative measures will be recorded with a simple questionnaire. Depending on the time of the experiments, we assume that people might also not be hungry enough to be prompted to have something to eat, which might disturb the results. We do plan however to measure whether the robot will remind someone of their hunger and have them eat.
++We measured the effectiveness of the discourse, both physically and emotionally. Our quantitative measure was whether the person ate the lunch they were supposed to have eaten, and the qualitative measure was the emotions that the PwD experienced before, during, and after the interaction. The qualitative measures were recorded with a simple questionnaire. Some people were not hungry enough to be prompted to have something to eat, which disturbed the results. However we did measure whether the robot reminded someone of their hunger and if they ate.
  == Procedure ==
--* Welcome Participants and explain what they are going to be doing.
--* Have them sign the permission form.
--* Participants complete a questionnaire(A) regarding their emotional state (control).
--* Have version A of interaction with the robot.
--* Complete questionnaire(extended version).
++The procedure was conducted as follows:
++
++1. Welcome participant and explain what they are going to be doing.
++2. Have them sign the permission form.
++* Complete questionnaire 1 regarding their emotional state (control).
++* Have an interaction with version A of the robot.
++* Complete questionnaire 2 (extended version).
  * Have a short interview during downtime (prepared questions).
--* Have version B of interaction with the robot.
--* Complete questionnaire(extended version).
++* Have an interaction with version B of the robot.
++* Complete questionnaire 3 (extended version).
  * Have a short interview during downtime (prepared questions).
  == Material ==
@@ -86,38 +86,69 @@
  = Results =
--== Mood ==
++The results were gathered from 19 personnel, all of whom interacted first with one version of the robot and then the other. 10 of the participants interacted first with the simple version, nine having their first interaction with the advanced version.
++== Eating ==
++
  {{html}}
--<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/MoodChangeDumb.png?rev=1.1" alt="Measured moods and changes for the simple robot" style="display:block;margin-left:auto;margin-right:auto;width:750"/>
++<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/EatingComp.png?rev=1.1" alt="Results on the eating of the test personas" style="display:block;margin-left:auto;margin-right:auto" width=750/>
  {{/html}}
  (% style="text-align:center" %)
--Figure: Median measured moods for the simple version of the robot
++Figure 1: Results on the eating of the test personas during the experiment
++Simple robot:
++
++* 16% ate
++* 33% of those would not have eaten without the robot
++
++Advanced robot:
++
++* 32% ate
++* 67% of those would not have eaten without the robot
++
++== Music ==
++
  {{html}}
--<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/MoodChangeSmart.png?rev=1.1" alt="Measured moods and changes for the advanced version of the robot" style="display:block;margin-left:auto;margin-right:auto;width:750"/>
++<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/MusicEnjoyable.png?rev=1.1" alt="Effects of music on the test personnel" style="display:block;margin-left:auto;margin-right:auto" width=1250/>
  {{/html}}
  (% style="text-align:center" %)
--Figure: Median measured moods for the advanced version of the robot
++Figure 2: Answers of the test personas regarding music
++
++== EVEA (Mood) ==
++
++{{html}}
++<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/MoodChangeDumb.png?rev=1.1" alt="Measured moods and changes for the simple robot" style="display:block;margin-left:auto;margin-right:auto" width=750/>
++{{/html}}
++
  (% style="text-align:center" %)
--Table: Wilcoxon signed rank test results for hypothesis that the mood changed during the interaction with the simple robot
++Figure 3: Median measured moods for the simple version of the robot
++{{html}}
++<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/MoodChangeSmart.png?rev=1.1" alt="Measured moods and changes for the advanced version of the robot" style="display:block;margin-left:auto;margin-right:auto" width=750/>
++{{/html}}
++
++(% style="text-align:center" %)
++Figure 4: Median measured moods for the advanced version of the robot
++
++(% style="text-align:center" %)
++Table 1: Wilcoxon signed rank test results for the hypothesis that the mood changed during the interaction with the simple robot
++
  |=Mood|=Happiness|=Anxiety|=Sadness|=Anger
  |Statistic|37|5|4|14
  |P-value|0.54|0.01|0.01|0.45
  (% style="text-align:center" %)
--Table: Wilcoxon signed rank test results for hypothesis that the mood changed during the interaction with the advanced robot
++Table 2: Wilcoxon signed rank test results for the null hypothesis that the mood changed during the interaction with the advanced robot
  |=Mood|=Happiness|=Anxiety|=Sadness|=Anger
--|Statistic|37|11|2|17
--|P-value|0.54|0.01|0.01|0.45
++|Statistic|32|11|2|17
++|P-value|0.18|0.01|0.01|0.45
  (% style="text-align:center" %)
--Table: Wilcoxon signed rank test results for hypothesis that the mood decreased during the interaction with the simple robot
++Table 3: Wilcoxon signed rank test results for the null hypothesis that the mood decreased during the interaction with the simple robot
  |=Mood|=Anxiety|=Sadness|=Anger
  |Statistic|81|53|29
@@ -124,7 +124,7 @@
  |P-value|0.01|0.00|0.23
  (% style="text-align:center" %)
--Table: Wilcoxon signed rank test results for hypothesis that the mood decreased during the interaction with the advanced robot
++Table 4: Wilcoxon signed rank test results for the null hypothesis that the mood decreased during the interaction with the advanced robot
  |=Mood|=Anxiety|=Sadness|=Anger
  |Statistic|32|149|52
@@ -131,7 +131,7 @@
  |P-value|0.00|0.01|0.07
  (% style="text-align:center" %)
--Table: Wilcoxon signed rank test results for hypothesis that the mood increased during the interaction with the simple robot
++Table 5: Wilcoxon signed rank test results for the null hypothesis that the mood increased during the interaction with the simple robot
  |=Mood|=Happiness
  |Statistic|37
@@ -138,49 +138,70 @@
  |P-value|0.27
  (% style="text-align:center" %)
--Table: Wilcoxon signed rank test results for hypothesis that the mood increased during the interaction with the advanced robot
++Table 6: Wilcoxon signed rank test results for the null hypothesis that the mood increased during the interaction with the advanced robot
  |=Mood|=Happiness
  |Statistic|32
  |P-value|0.09
--== Music ==
++(% style="text-align:center" %)
++Table 7: Wilcoxon signed rank test results for the hypothesis that the mood changes with the simple and advanced robots during the interaction are different
++|=Mood|=Happiness|=Anxiety|=Sadness|=Anger
++|Statistic|92|49|85|69
++|P-value|0.92|0.07|0.71|0.31
++
++== Godspeed ==
++
  {{html}}
--<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/MusicEnjoyable.png?rev=1.1" alt="Effects of music on the test personnel" style="display:block;margin-left:auto;margin-right:auto;width:750"/>
++<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/friendly-hist.png?rev=1.1" alt="Effects of music on the test personnel" style="display:block;margin-left:auto;margin-right:auto" width=750/>
  {{/html}}
  (% style="text-align:center" %)
--Figure: Answers of the test personas regarding music
++Figure 5: Answers to the statement 'I thought the robot was friendly'
--== Music ==
--
  {{html}}
--<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/MusicEnjoyable.png?rev=1.1" alt="Effects of music on the test personnel" style="display:block;margin-left:auto;margin-right:auto;width:750"/>
++<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/pleasant-hist.png?rev=1.1" alt="Answers to the statement 'I thought the robot was pleasant'." style="display:block;margin-left:auto;margin-right:auto" width=750/>
  {{/html}}
  (% style="text-align:center" %)
--Figure: Answers of the test personas regarding music
++Figure 6: Answers to the statement 'I thought the robot was pleasant'
--== Eating ==
--
  {{html}}
--<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/EatingComp.png?rev=1.1" alt="Results on the eating of the test personas" style="display:block;margin-left:auto;margin-right:auto;width:750"/>
++<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group01/download/Test/WebHome/godspeed-barchart.png?rev=1.1" alt="Godspeed questionnaire median comparison'." style="display:block;margin-left:auto;margin-right:auto" width=750/>
  {{/html}}
  (% style="text-align:center" %)
--Figure: Results on the eating of the test personas during the experiment
++Figure 7: Median measured Godspeed questionnaire dimensions
--- TODO: Add "I thought the robot was pleasant." and "I thought the robot was friendly" figures
++(% style="text-align:center" %)
++Table 8: Wilcoxon signed rank test results for the null hypothesis that the advanced robot scored higher in the perceived dimensions
++
++|=Dimension|=Likeability|=Intelligence
++|Statistic|36|70
++|P-value|0.01|0.17
++
  = Discussion =
++Analysis the results surfaced some minor issues in the experiment, such as the lack of comparison with two robots of similar features, with and without music. Also the practical limitations in the setup, such as the lack of different food options and some participants being aware of the design goals of the prototype could have interfered with the natural flow of the intercourse. With these limitations, the research method was successful in extracting differences within the robots and brought up additional directions for future research.
++The most interesting direction for future research would be the longer term studying of the effect of mealtime reminders on the health of the test subjects. The longer term health study would uncover the effect on eating frequency and the development of the relationship with the robot, for example would the test subjects that were first excited about the novel interaction with the robot, develop negative feelings about the supervision that the robot is conducting into their personal life.
++Another topic to study is the differences with and without music. The effects of music could be studied with the music tailored to personal taste and all versions of the robot with and without the music playback included in the interaction. This would allow to pinpoint the effects of music, without the other features causing variance.
  = Conclusions =
--The experiment showed promising results and encourages more research into the subject.
++From the results we can see that the more advanced robot shows advantages over the simple version in many categories. Hints of better performance in other categories can be seen, but no conclusions should be drawn from the ones that lack the statistical significance.
++In improving the eating, it seems that both robots have limited success in causing the people to eat as seen in Figure 1, they could cause the patients to eat more regularly, if triggered by timers or other suitable systems. It also seems that the advanced robot is better in the reminding, by a slight margin. However, the long term effects of reminding should be researched more to conclude whether the usage of the demonstrated robot platform or similar would cause the patients to eat more regularly. It is also unclear how the test setup and the limited choice of food affected the eating.
++
++Based on the answers of the participants regarding music seen in Figure 2, it seems that most of them were either indifferent or liked the music. Also, as the test personnel find the advanced robot more likeable with a 5% confidence limit (Table 7), and the advanced version was the only version with music, it seems likely that the music does make the interaction more pleasant for the personas. However, some of the likeability might be due to the other advanced features of the robot and thus more research is needed to conclude the effect of the music.
++
++The EVEA and partial Godspeed result can be seen in Figures 3-7 and Tables 1-8. The results show that with reasonable confidence (5% confidence limit), both versions of the robot decreased sadness and anxiety in the test personas. Hints are shown (10% confidence limit) that the advanced robot also decreases feelings of anger and increases happiness, while the simple robot fails to show similar results. However, in Table 7 we can see that the statistical differences in the mood differences during the interactions with the different versions are not highly significant.
++
++A Wilcoxon signed rank test for the partial Godspeed test shows in Table 8 that with high confidence (1% confidence limit), the intelligent robot is more likeable in comparison to the simple robot. With these results it is likely that the more advanced robot is slightly preferrable and the personas might experience less negative emotions after the interaction with the robots, but it is slightly unclear if the effect is more powerful with the advanced robot.
++
++
  = Appendix =
  == Experiment introduction for participants ==

godspeed-barchart.png

Author

...	...	@@ -1,0 +1,1 @@
	1	+XWiki.VSaikkonen

Size

...	...	@@ -1,0 +1,1 @@
	1	+15.9 KB

Content