Wiki source code of Test

Version 10.3 by Simran Karnani on 2022/03/26 13:51

Hide last authors
Clara Stiller 2.4 1 The best way to test our prototype would be a study with persons with dementia. Still, testing the robot in a real environment would be very time-consuming, because it is not predictable if and when persons with dementia start wandering. That is out of scoop for our project.
2 However, we want to get a first impression of how realistic and convincing the provided dialogues and suggested activities are. In a small study with students, who play the role of having dementia, we are observing the interaction with the robot and want to find out how effective it is in preventing people from wandering.
3
Clara Stiller 2.5 4
Xin Wan 1.2 5 = Problem statement and research questions =
Clara Stiller 4.1 6
Simran  Karnani 1.4 7 **Goal**: How effective is music and dialogue in preventing people with dementia from wandering?
Bart Vastenhouw 1.1 8
Clara Stiller 2.7 9 **Research Questions (RQ):**
Simran  Karnani 1.4 10
Simran  Karnani 10.3 11 1. What percentage of people are prevented from going out unsupervised? (Quantitative) (CL01, CL05)
12 2. How does the interaction change the participant's mood? (CL02)
13 3. Can the robot respond appropriately to the participant's intention? (CL03)
14 4. How do the participants react to the music? (CL04)
15 5. Does the activity that the robot suggests prevent people from wandering/ leaving? (CL06)
Simran  Karnani 1.4 16
Vishruty Mittal 10.1 17 //Future research questions//
Simran  Karnani 10.3 18 1. Does the interaction with Pepper make PwD come back to reality? (CL07)
19 2. Does the interaction with Pepper make PwD feel he/she is losing freedom? (CL08)
20 3. Does preventing the participant from going out alone make them feel dependent? (CL09)
Vishruty Mittal 9.1 21
Vishruty Mittal 10.1 22
Xin Wan 1.2 23 = Method =
Clara Stiller 4.1 24
Clara Stiller 2.6 25 A between-subject study with students who play the role of having dementia. Data will be collected by observing the participant's body language and the way that they're responding to Pepper.
Bart Vastenhouw 1.1 26
Xin Wan 1.2 27 == Participants ==
Clara Stiller 4.1 28
Clara Stiller 2.7 29 20 students who play the role of having dementia. They will be divided into two groups. One group will be interacting with the intelligent (group 1) robot while the other will interact with the unintelligent robot (group 2).
Vishruty Mittal 10.1 30 It is assumed that all participants are living at the same care centre.
Simran  Karnani 1.9 31 Before they start, they can choose how stubborn they want to be and how progressed their stage of dementia is.
Bart Vastenhouw 1.1 32
Xin Wan 1.2 33 == Experimental design ==
Clara Stiller 4.1 34
Clara Stiller 2.6 35 The first question collects quantitative data, where the number of people who were stopped will be counted.
Vishruty Mittal 10.1 36 The other RQs are exploratory research.
Clara Stiller 4.1 37
Clara Stiller 2.6 38 1. Observe the participant's mood and see how the conversation goes. Observe the level of aggression (tone, volume, pace)
Vishruty Mittal 10.1 39 1. Observe whether the mood is improved and the decision has been changed.
Simran  Karnani 1.11 40 1. Observe how natural the conversation is. (conversation makes sense)
Vishruty Mittal 10.1 41 1. Participants fill out questionnaires.
Bart Vastenhouw 1.1 42
Clara Stiller 4.1 43 == Tasks ==
Clara Stiller 2.6 44
Clara Stiller 2.7 45 Because our participants only play the role of having dementia, we will give them a level of stubbornness/ willpower with they are trying to leave. We try to detect this level with the robot.
46 Participants from group 1 (using intelligent robot) will also be given one of the reasons to leave, listed below:
Clara Stiller 4.1 47
Clara Stiller 2.7 48 1. going to the supermarket
49 1. going to the office
50 1. going for a walk
51
Clara Stiller 2.8 52 After this preparation, the participant is told to (try to) leave the building. The participant and robot have an interaction where the robot is trying to convince the participant to stay inside.
Bart Vastenhouw 1.1 53
54
Xin Wan 1.2 55 == Measures ==
Clara Stiller 4.1 56
Simran  Karnani 1.14 57 We will be measuring this physically and emotionally.
58 Physically: whether the participant was stopped from leaving the building or not.
59 Emotionally: evaluate their responses to the robot and observe their mood before and after the interaction.
Bart Vastenhouw 1.1 60
61
Xin Wan 1.2 62 == Procedure ==
Clara Stiller 4.1 63
Clara Stiller 3.1 64 {{html}}
65 <!-- Your HTML code here -->
66 <table width='100%'>
67 <tr>
68 <th width='50%'>Group 1</th>
69 <th width='50%'>Group 2</th>
70 </tr>
71 <tr>
72 <td>intelligent robot</td>
73 <td>unintelligent robot</td>
74 </tr>
75 <tr>
76 <td>
Clara Stiller 4.2 77 1. Starts with a short briefing on what we expect from the participant<br>
78 2. Let them fill out the informed consent form<br>
Clara Stiller 5.2 79 3. Tell them their level of stubbornness and reason to leave<br>
80 4. Let the user interact with the robot<br>
81 5. While user is interacting, we will be observing the conversation and the participant's moods <br>
82 6. Let user fill out the questionnaire about their experience after the interaction
Clara Stiller 3.1 83 </td>
84 <td>
Clara Stiller 4.2 85 1. Starts with a short briefing on what we expect from the participant<br>
86 2. Let them fill out the informed consent form<br>
87 3. Tell them their level of stubbornness<br>
88 4. Let the user interact with the robot<br>
89 5. Let user fill out the questionnaire about their experience after the interaction<br>
Clara Stiller 3.1 90 </td>
91 </tr>
92 </table>
Bart Vastenhouw 1.1 93
Clara Stiller 3.1 94 {{/html}}
Clara Stiller 2.6 95
Xin Wan 1.2 96 == Material ==
Clara Stiller 4.1 97
Simran  Karnani 1.16 98 Pepper, laptop, door, and music.
Bart Vastenhouw 1.1 99
100
Xin Wan 1.2 101 = Results =
Clara Stiller 6.2 102
Clara Stiller 5.5 103 **Results from Questionnaire:**
Bart Vastenhouw 1.1 104
Clara Stiller 7.2 105 1. Condition 1 - intelligent Prototype:
106 8 out of 11 Participants answered, that they don't know the music that has been played. If we told them afterward the title of the song, most participants do know the song. Why didn't they recognize it during the interaction?
107 This can have two reasons: The part of the song we pick was too short to be recognized or not the most significant part of the song. For example, the beginning of "escape - the pina colada song" is not as well known as its chorus. Another reason could be, that the participant was distracted or confused by the robot and therefore couldn't carefully listen to the music.
Clara Stiller 8.1 108 Only 4 out of 11 people agreed, that the music fits the situation. One of our claims, to use music that fits the situation or place, is therefore not reached and the music didn't have the intended effect. Even though we carefully choose the music and discussed a lot about our choice, it was hard to find music that different people connect with a certain place or activity. An approach to improve this could be using an individual playlist for each participant.
Clara Stiller 7.2 109
110
Clara Stiller 6.2 111 **Feedback from Participants:**
Bart Vastenhouw 1.1 112
Clara Stiller 6.2 113 **Observations:**
114 Problems that occurred during the evaluation
Bart Vastenhouw 1.1 115
Clara Stiller 6.2 116 1. lots of difficulties with speech recognition:
117 1.1. even though the participant said one of the expected words, pepper understood it wrong and continued with a wrong path
118 1.2. If the participant started to talk before pepper was listening (eyes turning blue), it misses a "yes" or "no" at the beginning of the sentence, which causes misunderstandings.
119 1. problems with face detection
120 2.1. due to bad light face was not recognized
121 2.2. if the participant passes pepper from the side, the face was not recognized. Therefore, we told people to walk from the front towards pepper. In most cases that helped detect the face.
122 2.3. face detection doesn't work with face masks. This could lead to huge problems in the usage of pepper in care homes.
Clara Stiller 5.5 123
Clara Stiller 7.1 124 One of the most frequent and noticeable reactions from participants was **confusion**. This feeling was caused by two main factors:
125 misunderstandings from speech recognition which leads to unsuitable answers from pepper, as well as the unsuitable environment and setting of our evaluation.
Clara Stiller 6.2 126 The reasons for failure in speech recognition are listed above. An unsuitable answer can e.g. be an argument to stay inside, that doesn't fit the participant's reason to leave. Also, some people told in a long sentence that they don't like the provided activity and still want to leave. If the speech recognition fails in this case and pepper understood you would like to do the activity, it seems like it encourages you to leave, instead of doing the activity. This leads to the total opposite of our intention.
Clara Stiller 7.1 127 Furthermore, we found out, that our prototype doesn't fit in the environment of the lab. We encourage the participant to do some activities, that they can't do in the lab environment (go to the living room, have a coffee or do a puzzle). If the robot tells asks you if you want to do the activity, most people don't know how to react and are insecure about how to answer. Participants "freeze" in front of the robot or just left the room.
Clara Stiller 6.2 128
Clara Stiller 7.2 129
130
Xin Wan 1.2 131 = Discussion =
Bart Vastenhouw 1.1 132
133
134
Xin Wan 1.2 135 = Conclusions =