Skip to Content

Changes for page Test

Last modified by Clara Stiller on 2022/04/05 13:44
From version Icon 67.2 Icon
edited by Vishruty Mittal
on 2022/04/02 12:58
Change comment: There is no comment for this version
To version Icon 53.1 Icon
edited by Sayak Mukherjee
on 2022/04/02 02:40
Change comment: There is no comment for this version

Summary

Details

Icon Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.Vishruty
1 +XWiki.SayakMukherjee
Content
... ... @@ -1,4 +1,5 @@
1 -Evaluation is an iterative process where the initial iterations focus on examining if the proposed idea is working as intended. Therefore, we want to first understand how realistic and convincing the provided dialogues and suggested activities are, and would they be able to prevent people from wandering. To examine this, we conduct a small pilot study with students, who role-play having dementia. We then observe their interaction with Pepper to examine the effectiveness of our dialog flow in preventing people from wandering.
1 +The best way to test our prototype would be a study with persons with dementia. However, testing the robot in a real environment would be very time-consuming, because it is not predictable if and when people with dementia start wandering. Hence it is out of the scope of this project.
2 +Nevertheless, we wanted to get a first impression of how realistic and convincing the provided dialogues and suggested activities are. In a small study with students, who play the role of having dementia, we are observing the interaction with the robot with an aim to find out how effective it is in preventing people from wandering.
2 2  
3 3  
4 4  = Problem statement and research questions =
... ... @@ -23,37 +23,35 @@
23 23  
24 24  = Method =
25 25  
26 -A between-subject study with students who play the role of having dementia. Data will be collected with a questionnaire that participants fill out before and after interacting with Pepper. The questionnaire captures different aspects of the conversation along with their mood before and after the interaction with Pepper.
27 +A between-subject study with students who play the role of having dementia. Data will be collected with a questionnaire (before and after participation), observing the participant's body language and the way that they're responding to Pepper.
27 27  
28 -For our between-subject study, our independent variable is Pepper trying to distract the users by mentioning different activities along with the corresponding music. Through this, we want to measure the effectiveness of music and activities in preventing people from leaving the care home, which is thereby our dependent variable. So we developed 2 different prototype designs-
29 -
30 -Design X - It is the full interaction flow where Pepper suggests activities and uses music to distract people from leaving.
31 -Design Y - It is the control condition where pepper simply tries to stop people from leaving by physically keeping its hand on the door.
32 -
33 33  == Participants ==
34 34  
35 -17 students who play the role of having dementia. They will be divided into two groups. One group (11 participants) will be interacting with design X (group 1) robot while the other group (6 students) will interact with the design Y (group 2).
31 +18 students who play the role of having dementia. They will be divided into two groups. One group (11 participants) will be interacting with the intelligent (group 1) robot while the other group (7 students) will interact with the unintelligent robot (group 2).
36 36  It is assumed that all participants are living at the same care center.
33 +Before they start, they can choose how stubborn they want to be and where they want to go.
37 37  
38 38  == Experimental design ==
39 39  
40 -**Before Experiment:**
37 +All questions collect quantitative data, using a 5 point Likert scale wherever applicable.
41 41  
42 -We will explain to the participants the goal of this experiment and what do they need to do to prevent ambiguity. Therefore, as our participants are students and only playing the role of having dementia, we will give them a level of stubbornness/ willpower with which they are trying to leave the care home.
43 -Participants will also be given a reason to leave, from the below list:
39 +1. Observe the participant's mood and see how the conversation goes. Observe the level of aggression (tone, volume, pace)
40 +1. Observe whether the mood is improved and the decision has been changed.
41 +1. Observe how natural the conversation is. (conversation makes sense)
42 +1. Participants fill out questionnaires.
44 44  
45 -* going to the supermarket
46 -* going to the office
47 -* going for a walk
44 +== Tasks ==
48 48  
49 -After this preparation, the participant fills a part of the questionnaire.
46 +Because our participants only play the role of having dementia, we will give them a level of stubbornness/ willpower with they are trying to leave. We try to detect this level with the robot.
47 +Participants from group 1 (using intelligent robot) will also be given one of the reasons to leave, listed below:
50 50  
51 -**Experiment:**
52 -The participant begins interacting with Pepper who is standing near the exit door. The participant and robot have an interaction where the robot is trying to convince him/her to stay inside.
49 +1. going to the supermarket
50 +1. going to the office
51 +1. going for a walk
53 53  
54 -**After Experiment:**
55 -After the participant finishes interacting with Pepper, he/she will be asked to fill out the remaining questionnaire. Almost all the questions in the questionnaire collect quantitative data, using a 5 point Likert scale. The questionnaire also used images from Self Assessment Manikin (SAM) so that user can self attest to their mood before and after their interaction with Pepper.
53 +After this preparation, the participant is told to (try to) leave the building. The participant and robot have an interaction where the robot is trying to convince the participant to stay inside.
56 56  
55 +
57 57  == Measures ==
58 58  
59 59  We will be measuring this physically and emotionally.
... ... @@ -100,8 +100,8 @@
100 100  
101 101  Pepper, laptop, door, and music.
102 102  
103 -= Results =
104 104  
103 += Results =
105 105  {{html}}
106 106  <!--=== Comparison between intelligent (cond. 1) and less intelligent (cond. 2) prototype ===
107 107  
... ... @@ -158,13 +158,19 @@
158 158  Furthermore, we found out, that our prototype doesn't fit in the environment of the lab. We encourage the participant to do some activities, that they can't do in the lab environment (go to the living room, have a coffee or do a puzzle). If the robot tells asks you if you want to do the activity, most people don't know how to react and are insecure about how to answer. Participants "freeze" in front of the robot or just left the room. -->
159 159  {{/html}}
160 160  
161 -=== RQ1: Are people convinced not to go out unsupervised? ===
160 +1. What percentage of people are prevented from going out unsupervised? (Quantitative) (CL01, CL05)
161 +1. How does the interaction change the participant's mood? (CL02)
162 +1. Can the robot respond appropriately to the participant's intention? (CL03)
163 +1. How do the participants react to the music? (CL04)
164 +1. Does the activity that the robot suggests prevent people from wandering/ leaving? (CL06)
165 +1. Can pepper identify and catch the attention of the PwD?
162 162  
167 +=== RQ1: Are people convinced not to go out unsupervised? ===
163 163  {{html}}
164 164  <table style="width: 100%">
165 165  <tr>
166 166  <td style="width: 50%">
167 -<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/RQ1.jpg?height=250&rev=1.1" />
172 +<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/Storyboard_1.png?height=750&rev=1.1" />
168 168  </td>
169 169  <td>
170 170  Comment on the graph
... ... @@ -174,12 +174,11 @@
174 174  {{/html}}
175 175  
176 176  === RQ2: How does the interaction change the participant's mood? ===
177 -
178 178  {{html}}
179 179  <table style="width: 100%">
180 180  <tr>
181 181  <td style="width: 50%">
182 -<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/RQ2.jpg?height=250&rev=1.1" />
186 +<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/Storyboard_1.png?height=750&rev=1.1" />
183 183  </td>
184 184  <td>
185 185  Comment on the graph
... ... @@ -189,12 +189,11 @@
189 189  {{/html}}
190 190  
191 191  === RQ3: Can the robot respond appropriately to the participant's intention? ===
192 -
193 193  {{html}}
194 194  <table style="width: 100%">
195 195  <tr>
196 196  <td style="width: 50%">
197 -<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/RQ3.jpg?height=250&rev=1.1" />
200 +<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/Storyboard_1.png?height=750&rev=1.1" />
198 198  </td>
199 199  <td>
200 200  Comment on the graph
... ... @@ -204,12 +204,11 @@
204 204  {{/html}}
205 205  
206 206  === RQ4: How do the participants react to the music? ===
207 -
208 208  {{html}}
209 209  <table style="width: 100%">
210 210  <tr>
211 211  <td style="width: 50%">
212 -<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/RQ4.jpg?height=250&rev=1.1" />
214 +<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/Storyboard_1.png?height=750&rev=1.1" />
213 213  </td>
214 214  <td>
215 215  Comment on the graph
... ... @@ -219,12 +219,11 @@
219 219  {{/html}}
220 220  
221 221  === RQ5: Does the activity that the robot suggests prevent people from wandering/ leaving? ===
222 -
223 223  {{html}}
224 224  <table style="width: 100%">
225 225  <tr>
226 226  <td style="width: 50%">
227 -<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/RQ5.jpg?height=250&rev=1.1" />
228 +<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/Storyboard_1.png?height=750&rev=1.1" />
228 228  </td>
229 229  <td>
230 230  Comment on the graph
... ... @@ -234,12 +234,11 @@
234 234  {{/html}}
235 235  
236 236  === RQ6: Can pepper identify and catch the attention of the PwD? ===
237 -
238 238  {{html}}
239 239  <table style="width: 100%">
240 240  <tr>
241 241  <td style="width: 50%">
242 -<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/RQ6.jpg?height=250&rev=1.1" />
242 +<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/Storyboard_1.png?height=750&rev=1.1" />
243 243  </td>
244 244  <td>
245 245  Comment on the graph
... ... @@ -249,12 +249,11 @@
249 249  {{/html}}
250 250  
251 251  === Reliabity Scores ===
252 -
253 253  {{html}}
254 254  <table style="width: 100%">
255 255  <tr>
256 256  <td style="width: 50%">
257 -<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/RelScores.jpg?height=250&rev=1.1" />
256 +<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/Storyboard_1.png?height=750&rev=1.1" />
258 258  </td>
259 259  <td>
260 260  Comment on the graph
... ... @@ -263,7 +263,7 @@
263 263  </table>
264 264  {{/html}}
265 265  
266 -= Limitation =
265 += Limitation=
267 267  
268 268  * **Lab Environment**: The lab environment is different from a care home, the participants found it difficult to process the suggestions made by Pepper. For example, if Pepper asked someone to visit the living room, it created confusion among the participants regarding their next action.
269 269  
... ... @@ -274,7 +274,6 @@
274 274  * **Face Detection**: The face recognition module within Pepper is also rudimentary in nature. It can not detect half faces are when participants approach from the side. Adding to the problem, the lighting condition in the lab was not sufficient for the reliable functioning of the face recognition module. Hence Pepper failed to notice the participant in some cases and did not start the dialogue flow.
275 275  
276 276  = Conclusions =
277 -
278 278  * People who liked the activity tend to stay in
279 279  * People who knew the music found it more fitting
280 280  * People are more convinced to stay in with the intelligent prototype
... ... @@ -283,8 +283,8 @@
283 283  * Experiment with personalization
284 284  
285 285  = Future Work =
286 -
287 287  * **Personalisation**: Personalize music, and activity preferences according to the person interacting with Pepper.
288 288  * **Robot Collaboration**: Collaborate with other robots such as Miro to assist a person with dementia while going for a walk instead of the caretaker.
289 289  * **Recognise Person**: For a personalised experience, it is essential that Pepper is able to identify each person based on an internal database.
290 290  * **Fine Tune Speech Recognition**: Improvements are necessary for the speech recognition module before the actual deployment of the project in a care home. Additionally, support for multiple languages can be considered to engage with non-English speaking people.
288 +
Icon XWiki.XWikiComments[1]
Comment
... ... @@ -1,1 +1,1 @@
1 -Add claim number with RQ6 once claims are fixed back
1 +Modified Research Questions so that they align with claims
Date
... ... @@ -1,1 +1,1 @@
1 -2022-04-02 11:44:44.339
1 +2022-03-26 13:50:45.49
Icon XWiki.XWikiComments[2]
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.Vishruty
Comment
... ... @@ -1,1 +1,0 @@
1 -Check if RQ and claims align with each other(CL numbers) once claims are reverted.
Date
... ... @@ -1,1 +1,0 @@
1 -2022-04-02 11:45:38.960