Skip to Content

Changes for page Test

Last modified by Laura Ottevanger on 2022/04/05 14:02
From version Icon 20.1 Icon
edited by Rohan Sobha
on 2022/03/14 18:05
Change comment: Moved comment thread to comments instead of page
To version Icon 40.3 Icon
edited by Doreen Mulder
on 2022/04/04 13:48
Change comment: There is no comment for this version

Summary

Details

Icon Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.rsobha
1 +XWiki.DoreenMulder
Content
... ... @@ -1,3 +1,5 @@
1 +In this section, we will introduce our experimental test set up and motivate our research questions.
2 +
1 1  = Problem statement and research questions =
2 2  
3 3  People with dementia face challenges that were not present in their life before being affected by their condition. This is also influenced by their living situation, since they may live in care homes or dementia centers. To ensure that people with dementia stay safe, activities are usually lead by a caretaker. However, staff has limited time available to spend with each individual person. Group activities or caretaker-lead activities may affect the feeling of autonomy of the person with dementia. Autonomy is one of the key psychological needs described in self-determination theory, which describes how feeling fulfilled is influenced by how much a person feels in control of their own actions. It is important to the mental well-being of people with dementia to keep a sense of autonomy.
... ... @@ -12,20 +12,22 @@
12 12  
13 13  = Method =
14 14  
15 -To this effect, we would like to invite users to participate in a usability study. In the study, we will assess our research question by dividing the participants into two groups: one with a //caretaker// and one with a //MiRo//. The former group will enact the activity of walking in the current setting and the latter group will do the equivalent with the robot. Afterwards, both groups are assessed by means of Likert-scales to measure the increase.
17 +To this effect, we would like to invite users to participate in a usability study. In the study, we will assess our research question by dividing the participants into two groups: one with a //caretaker// and one with a //MiRo//. The former group will enact the activity of walking in the current setting and the latter group will do the equivalent with the robot. Afterwards, both groups are assessed by means of Likert-scales to measure the increase. The difference in scores between the caretaker and the MiRo (WAF) setting will then be subjected to a Wilcoxon [[(Conover, 1998, p. 250)>>https://www-wiley-com.tudelft.idm.oclc.org/en-nl/Practical+Nonparametric+Statistics%2C+3rd+Edition-p-9780471160687]] test to determine whether the increase is statistically significant. If the significance has been demonstrated, then we can reject the null hypothesis: walking with the MiRo will not provide an increased feeling of autonomy compared to walking with a caretaker.
16 16  
17 17  
18 18  == Participants ==
19 19  
20 20  For our study, we would have liked to invite people with dementia to participate. However, user testing with a vulnerable group as this one is difficult seeing the current circumstances with the COVID-19 pandemic.
21 -For this reason, we decided to design an experiment to be conducted on our fellow students. Students will be briefed on dementia. Then they take on the role of one of the personas, and roleplay a person with dementia. Through this roleplay, we aim to simulate a study with people with dementia as accurately as possible in the given situation.
23 +For this reason, we decided to design an experiment to be conducted on our fellow students. Students will be briefed on dementia. Then they take on the role of one of the personas, and roleplay a person with dementia. Through this roleplay, we aim to simulate a study with people with dementia as accurately as possible in the given situation. We will give them one page of the [[persona handout document>>https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group02/download/Test/WebHome/personas-handouts.pdf?rev=1.1]]
22 22  
23 23  == Experimental design ==
24 24  
25 -The main research question can be answered using a within-subject design in order to be able to measure the //increase// per subject. The learning effect will be mitigated, because the two juxtaposed settings are vastly different. So, there is no advantage for the participant to be subjected to two conditions serially.
27 +The main research question can be answered using a within-subject design in order to be able to measure the //increase// per subject. The learning effect will be mitigated, because the two juxtaposed settings are vastly different. So, there is no advantage for the participant to be subjected to two conditions serially. In addition to the questionnaire, notes will be taken during the experiment of the observations of the participant's interactions with WAF (Walking Assisting Friend).
26 26  
27 -The experiment will be recorded in the form of audio recording which are to be transcribed afterwards. Naturally, the recordings will be deleted subsequently to comply with GDPR.
29 +== Ethical Approval ==
28 28  
31 +Though the official ethical approval was handled by the course staff, we have also filled in the HREC Risk Assessment and Mitigation Plan form. The form can be found [[here>>https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group02/download/Test/WebHome/HREC-checklist-group-2.pdf]]. Before participating in the study, the participants signed the consent form found [[here>>https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group02/download/Test/WebHome/consent-group-2.pdf]]
32 +
29 29  == Tasks ==
30 30  
31 31  We break up our experiment into the following tasks:
... ... @@ -49,11 +49,8 @@
49 49  
50 50  == Procedure ==
51 51  
52 -There are two different situations that need to be tested.
53 -The first situation is walking with the Miro. The PwD is given the MiRo and the bracelet/leash, the MiRo has a pre-programmed route to walk. The PwD is asked to walk with the dog. While they are walking the dog, they are observed, preferably by cameras as to simulate the situation how the MiRo should be used. The emotional state of the PwD is actively monitored by looking at facial expressions. Furthermore, the researchers will actively check how easily the PwD interact with the MiRo to check the familiarity. When the PwD is done walking, some small questions will be asked to check the satisfaction, autonomy and familiarity.
56 +The procedure is that the person with dementia is walking with the Miro. The PwD is given the MiRo and the bracelet/leash, the MiRo has a pre-programmed route to walk. The PwD is asked to walk with the dog. While they are walking the dog, they are observed, preferably by cameras as to simulate the situation how the MiRo should be used. The emotional state of the PwD is actively monitored by looking at facial expressions. Furthermore, the researchers will actively check how easily the PwD interact with the MiRo to check the familiarity. When the PwD is done walking, some small questions will be asked to check the satisfaction, autonomy and familiarity.
54 54  
55 -The second situation is Snoezelen. The PwD will sit at a table, the MiRo will be put on the table. The researcher will either stay in the room or put up cameras to observe the PwD. The PwD will be encouraged to pet and play with the MiRo. During the Snoezelen, the emotional state of the PwD will be observed. The researchers will also observe how easily the PwD handles the MiRo. After the snoezelen, the PwD will be asked how they felt during the Snoezelen.
56 -
57 57  Functions like when an elderly person falls or when they walk the wrong direction will not forcibly be tested with PwD as this might create a dangerous situation. These functions will be tested in advance with the use of "Dogfeeding". Other people from the development-team will test these functions.
58 58  
59 59  == Material ==
... ... @@ -66,10 +66,58 @@
66 66  
67 67  = Results =
68 68  
70 +We asked participants about there sense of being in charge, happiness, safety and trust when walking with MiRo and with the caretaker. In the graphs below we show each sense compared between the two guiding agents.
69 69  
72 +{{html}}
73 +<table width='80%'>
74 +<tr>
75 +<td width='60%'>
76 +<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group02/download/Test/WebHome/In_charge_graph_comparison.jpg" width="500" height="398">
77 +</td>
78 +<td align="left">
79 +<p align="left">This comparison shows that when walking with a caretaker the participants felt mostly in charge, whereas when walking with MiRo the participants felt this less frequently.</p>
80 +</td>
81 +</tr>
82 +<tr>
83 +<td>
84 +<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group02/download/Test/WebHome/Happy_graph_comparison.jpg" alt="Feeling of being in charge" width="500" height="398" align="left">
85 +</td>
86 +<td>
87 +This comparison shows that both a caretaker or MiRo accompanied positive emotions.
88 +</td>
89 +</tr>
90 +<tr>
91 +<td>
92 +<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group02/download/Test/WebHome/Safe_graph_comparison.jpg" alt="Feeling of being in charge" width="500" height="398" align="left">
93 +</td>
94 +<td>
95 +This comparison shows a significant difference between walking with a caretaker and MiRo. A caretaker made the participants feel safe most of the time or prevented any unsafe feelings. MiRo conveyed only minimal feelings of safety with participants answering 'at times' most frequently.
96 +</td>
97 +</tr>
98 +<tr>
99 +<td>
100 +<img src="https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2022group02/download/Test/WebHome/Trustworthy_graph_comparison.jpg" alt="Feeling of trust" width="500" height="398" align="left">
101 +</td>
102 +<td>
103 +This comparison shows similar results to the safety comparison. A caretaker is reported to be mostly or completely trustworthy, while MiRo is trusted less as participants trusted it 'a little' or 'mostly'.
104 +</td>
105 +</tr>
106 +</table>
107 +{{/html}}
70 70  
109 +
110 +
71 71  = Discussion =
72 72  
113 +Through this test, we aim to answer the question: "Does walking with the MiRo increase the perceived autonomy of people with dementia, compared to walking under guidance of a caretaker?"
114 +We can conclude that in the current state of our prototype, this is not the case. The caretaker is preferred when asked if the user feels in charge, feels safe, and feels trustworthy. However, we note that the MiRo does slightly outperform the caretaker in regards to happiness. We think that further development is needed to accurately gauge if walking with a MiRo will eventually score equally or better compared to walking with a caretaker.
73 73  
116 +=== Inclusivity ===
74 74  
118 +In addition to the questionnaire, we conducted a short interview with our participant who is hard-of-hearing. Their study was conducted without them wearing their hearing aids.
119 +We asked them if they thought their experience would differ if they were wearing their hearing aids. Their answer was that the biggest thing was that in a real scenario, the wristband doesn't give you enough information about what to do. It only tells you that you are something wrong. They commented that if they were 83, they would not have enough direction from this, compared to someone who could hear sounds: MiRo could direct them back by barking at the user, for example. However, they mention that if you are severely hearing impaired, there is probably nothing you can do other than vibrating the wristband.
120 +We asked them how we could make it more inclusive. They mention that the robot is small and not very visible, which might cause problems for people with a sight impairment. People who are not familiar with walking dogs would also not naturally keep looking at MiRo. They recommended using a larger robot.
121 +
122 +
123 +
75 75  = Conclusions =
Icon HREC-checklist-group-2.pdf
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.ricardovogel
Size
... ... @@ -1,0 +1,1 @@
1 +67.1 KB
Content Icon
Icon Happy_graph_comparison.jpg
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.Harmen
Size
... ... @@ -1,0 +1,1 @@
1 +48.2 KB
Content Icon
Icon In_charge_graph_comparison.jpg
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.Harmen
Size
... ... @@ -1,0 +1,1 @@
1 +51.8 KB
Content Icon
Icon Safe_graph_comparison.jpg
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.Harmen
Size
... ... @@ -1,0 +1,1 @@
1 +48.1 KB
Content Icon
Icon Trustworthy_graph_comparison.jpg
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.Harmen
Size
... ... @@ -1,0 +1,1 @@
1 +52.2 KB
Content Icon
Icon consent-group-2.pdf
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.ricardovogel
Size
... ... @@ -1,0 +1,1 @@
1 +27.7 KB
Content Icon
Icon personas-handouts.pdf
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.ricardovogel
Size
... ... @@ -1,0 +1,1 @@
1 +1.4 MB
Content Icon