Skip to Content

Changes for page Test

Last modified by Clara Stiller on 2022/04/05 13:44
From version Icon 14.1 Icon
edited by Simran Karnani
on 2022/03/26 14:08
Change comment: There is no comment for this version
To version Icon 16.3 Icon
edited by Clara Stiller
on 2022/03/27 13:17
Change comment: There is no comment for this version

Summary

Details

Icon Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.SimranKarnani
1 +XWiki.ClaraStiller
Content
... ... @@ -1,4 +1,4 @@
1 -The best way to test our prototype would be a study with persons with dementia. Still, testing the robot in a real environment would be very time-consuming, because it is not predictable if and when persons with dementia start wandering. That is out of scoop for our project.
1 +The best way to test our prototype would be a study with persons with dementia. Still, testing the robot in a real environment would be very time-consuming, because it is not predictable if and when people with dementia start wandering. That is out of scoop for our project.
2 2  However, we want to get a first impression of how realistic and convincing the provided dialogues and suggested activities are. In a small study with students, who play the role of having dementia, we are observing the interaction with the robot and want to find out how effective it is in preventing people from wandering.
3 3  
4 4  
... ... @@ -26,12 +26,12 @@
26 26  
27 27  = Method =
28 28  
29 -A between-subject study with students who play the role of having dementia. Data will be collected by observing the participant's body language and the way that they're responding to Pepper.
29 +A between-subject study with students who play the role of having dementia. Data will be collected with a questionnaire (before and after participation), observing the participant's body language and the way that they're responding to Pepper.
30 30  
31 31  == Participants ==
32 32  
33 -18 students who play the role of having dementia. They will be divided into two groups. One group (11 participants) will be interacting with the intelligent (group 1) robot while the other group (7 student) will interact with the unintelligent robot (group 2).
34 -It is assumed that all participants are living at the same care centre.
33 +18 students who play the role of having dementia. They will be divided into two groups. One group (11 participants) will be interacting with the intelligent (group 1) robot while the other group (7 students) will interact with the unintelligent robot (group 2).
34 +It is assumed that all participants are living at the same care center.
35 35  Before they start, they can choose how stubborn they want to be and where they want to go.
36 36  
37 37  == Experimental design ==
... ... @@ -80,16 +80,17 @@
80 80  1. Starts with a short briefing on what we expect from the participant<br>
81 81  2. Let them fill out the informed consent form<br>
82 82  3. Tell them their level of stubbornness and reason to leave<br>
83 +4. Fill out question about current mood (in their role)<br>
83 83  4. Let the user interact with the robot<br>
84 -5. While user is interacting, we will be observing the conversation and the participant's moods <br>
85 +5. While user is interacting, we will be observing the conversation with the robot<br>
85 85  6. Let user fill out the questionnaire about their experience after the interaction
86 86  </td>
87 87  <td>
88 88  1. Starts with a short briefing on what we expect from the participant<br>
89 89  2. Let them fill out the informed consent form<br>
90 -3. Tell them their level of stubbornness<br>
91 -4. Let the user interact with the robot<br>
92 -5. Let user fill out the questionnaire about their experience after the interaction<br>
91 +4. Fill out question about current mood (in their role)<br>
92 +5. Let the user interact with the robot<br>
93 +6. Let user fill out the questionnaire about their experience after the interaction<br>
93 93  </td>
94 94  </tr>
95 95  </table>
... ... @@ -110,9 +110,19 @@
110 110  This can have two reasons: The part of the song we pick was too short to be recognized or not the most significant part of the song. For example, the beginning of "escape - the pina colada song" is not as well known as its chorus. Another reason could be, that the participant was distracted or confused by the robot and therefore couldn't carefully listen to the music.
111 111  Only 4 out of 11 people agreed, that the music fits the situation. One of our claims, to use music that fits the situation or place, is therefore not reached and the music didn't have the intended effect. Even though we carefully choose the music and discussed a lot about our choice, it was hard to find music that different people connect with a certain place or activity. An approach to improve this could be using an individual playlist for each participant.
112 112  
114 +1. Condition 2 - less intelligent prototype:
115 +Non of the participants who interacted with the less intelligent robot were prevented from leaving.
113 113  
114 -**Feedback from Participants:**
115 115  
118 +
119 +**Comparison between intelligent (cond. 1) and less intelligent (cond. 2) prototype**
120 +
121 +{{html}}
122 +<img src="/xwiki/wiki/sce2022group05/download/Test/WebHome/Stay_inside.svg? width="40" height="40"" />
123 +{{/html}}
124 +
125 +Non of the participants from condition 2 could be stopped from going out.
126 +
116 116  **Observations:**
117 117  Problems that occurred during the evaluation
118 118  
... ... @@ -129,10 +129,19 @@
129 129  The reasons for failure in speech recognition are listed above. An unsuitable answer can e.g. be an argument to stay inside, that doesn't fit the participant's reason to leave. Also, some people told in a long sentence that they don't like the provided activity and still want to leave. If the speech recognition fails in this case and pepper understood you would like to do the activity, it seems like it encourages you to leave, instead of doing the activity. This leads to the total opposite of our intention.
130 130  Furthermore, we found out, that our prototype doesn't fit in the environment of the lab. We encourage the participant to do some activities, that they can't do in the lab environment (go to the living room, have a coffee or do a puzzle). If the robot tells asks you if you want to do the activity, most people don't know how to react and are insecure about how to answer. Participants "freeze" in front of the robot or just left the room.
131 131  
143 +Condition 2:
144 +Participants assigned to condition 2 weren't convinced to leave. We saw, that most of them tried to continue talking to pepper when it raises its arm to block the door, even though it didn't listen. They were surprised by peppers reaction and asked for a reason why they are not allowed to leave. In order to have a natural conversation flow, the robot should provide an explanation for each scenario that tells why the person is not allowed to leave. This confirms that our approach, to give reason to stay inside, might be helpful to convince PwD to stay inside.
132 132  
133 -
134 134  = Discussion =
135 135  
136 136  
137 137  
150 +
151 +
152 +
138 138  = Conclusions =
154 +just some notes...
155 +* Evaluation is not significant enough to answer our main question/ goal, due to many issues with speech reco and misunderstandings
156 +* difference in effectiveness between cond 1 and cond 2, but not in all cases
157 +* Confusion + waiting for robot to listen -> bad for PwD, because they are already confused and might not remember to wait for the eyes to turn blue
158 +
Icon STay_inside.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.ClaraStiller
Size
... ... @@ -1,0 +1,1 @@
1 +26.3 KB
Content Icon
Icon Stay_inside.svg
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.ClaraStiller
Size
... ... @@ -1,0 +1,1 @@
1 +6.0 KB
Content
... ... @@ -1,0 +1,1 @@
1 +<svg width="1381" height="781" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" overflow="hidden"><defs><clipPath id="clip0"><rect x="1075" y="2251" width="1381" height="781"/></clipPath><clipPath id="clip1"><rect x="1075" y="2251" width="1379" height="778"/></clipPath><clipPath id="clip2"><rect x="1293" y="2421" width="1114" height="408"/></clipPath><clipPath id="clip3"><rect x="1075" y="2251" width="1379" height="778"/></clipPath><clipPath id="clip4"><rect x="1075" y="2251" width="1379" height="778"/></clipPath><clipPath id="clip5"><rect x="1075" y="2251" width="1379" height="778"/></clipPath><clipPath id="clip6"><rect x="1075" y="2251" width="1379" height="778"/></clipPath><clipPath id="clip7"><rect x="1075" y="2251" width="1379" height="778"/></clipPath><clipPath id="clip8"><rect x="1075" y="2251" width="1379" height="778"/></clipPath><clipPath id="clip9"><rect x="1075" y="2251" width="1379" height="778"/></clipPath><clipPath id="clip10"><rect x="1075" y="2251" width="1379" height="778"/></clipPath><clipPath id="clip11"><rect x="1075" y="2251" width="1379" height="778"/></clipPath><clipPath id="clip12"><rect x="1075" y="2251" width="1379" height="778"/></clipPath><clipPath id="clip13"><rect x="1075" y="2251" width="1379" height="778"/></clipPath><clipPath id="clip14"><rect x="1075" y="2251" width="1379" height="778"/></clipPath><clipPath id="clip15"><rect x="1075" y="2251" width="1379" height="778"/></clipPath><clipPath id="clip16"><rect x="1075" y="2251" width="1379" height="778"/></clipPath><clipPath id="clip17"><rect x="1075" y="2251" width="1379" height="778"/></clipPath><clipPath id="clip18"><rect x="1075" y="2251" width="1379" height="778"/></clipPath><clipPath id="clip19"><rect x="1075" y="2251" width="1379" height="778"/></clipPath></defs><g clip-path="url(#clip0)" transform="translate(-1075 -2251)"><rect x="1076" y="2252" width="1378" height="778" fill="#FFFFFF"/><g clip-path="url(#clip1)"><path d="M1293.5 2768.59 2403.5 2768.59M1293.5 2710.59 2403.5 2710.59M1293.5 2652.59 2403.5 2652.59M1293.5 2594.59 2403.5 2594.59M1293.5 2537.58 2403.5 2537.58M1293.5 2479.58 2403.5 2479.58M1293.5 2421.5 2403.5 2421.5" stroke="#D9D9D9" stroke-width="3.4375" stroke-linejoin="round" stroke-miterlimit="10" fill="none"/></g><g clip-path="url(#clip2)"><path d="M1369.04 2710.09 1439.05 2710.09 1439.05 2826 1369.04 2826ZM1592.05 2479.08 1661.05 2479.08 1661.05 2826 1592.05 2826ZM1814.06 2479.08 1883.06 2479.08 1883.06 2826 1814.06 2826ZM2036.07 2595.08 2106.07 2595.08 2106.07 2826 2036.07 2826ZM2258.07 2595.08 2328.08 2595.08 2328.08 2826 2258.07 2826Z" fill="#009999"/></g><g clip-path="url(#clip3)"><path d="M1293.5 2826.5 2403.5 2826.5" stroke="#D9D9D9" stroke-width="3.4375" stroke-linejoin="round" stroke-miterlimit="10" fill="none" fill-rule="evenodd"/></g><g clip-path="url(#clip4)"><text fill="#595959" font-family="Calibri,Calibri_MSFontService,sans-serif" font-weight="400" font-size="41" transform="matrix(1 0 0 1 1234.11 2838)">0</text></g><g clip-path="url(#clip5)"><text fill="#595959" font-family="Calibri,Calibri_MSFontService,sans-serif" font-weight="400" font-size="41" transform="matrix(1 0 0 1 1202.92 2780)">0,5</text></g><g clip-path="url(#clip6)"><text fill="#595959" font-family="Calibri,Calibri_MSFontService,sans-serif" font-weight="400" font-size="41" transform="matrix(1 0 0 1 1234.11 2722)">1</text></g><g clip-path="url(#clip7)"><text fill="#595959" font-family="Calibri,Calibri_MSFontService,sans-serif" font-weight="400" font-size="41" transform="matrix(1 0 0 1 1202.92 2664)">1,5</text></g><g clip-path="url(#clip8)"><text fill="#595959" font-family="Calibri,Calibri_MSFontService,sans-serif" font-weight="400" font-size="41" transform="matrix(1 0 0 1 1234.11 2606)">2</text></g><g clip-path="url(#clip9)"><text fill="#595959" font-family="Calibri,Calibri_MSFontService,sans-serif" font-weight="400" font-size="41" transform="matrix(1 0 0 1 1202.92 2549)">2,5</text></g><g clip-path="url(#clip10)"><text fill="#595959" font-family="Calibri,Calibri_MSFontService,sans-serif" font-weight="400" font-size="41" transform="matrix(1 0 0 1 1234.11 2491)">3</text></g><g clip-path="url(#clip11)"><text fill="#595959" font-family="Calibri,Calibri_MSFontService,sans-serif" font-weight="400" font-size="41" transform="matrix(1 0 0 1 1202.92 2433)">3,5</text></g><g clip-path="url(#clip12)"><text fill="#595959" font-family="Calibri,Calibri_MSFontService,sans-serif" font-weight="400" font-size="41" transform="matrix(1 0 0 1 1305.46 2891)">1 - Disagree</text></g><g clip-path="url(#clip13)"><text fill="#595959" font-family="Calibri,Calibri_MSFontService,sans-serif" font-weight="400" font-size="41" transform="matrix(1 0 0 1 1616.03 2891)">2</text></g><g clip-path="url(#clip14)"><text fill="#595959" font-family="Calibri,Calibri_MSFontService,sans-serif" font-weight="400" font-size="41" transform="matrix(1 0 0 1 1838.17 2891)">3</text></g><g clip-path="url(#clip15)"><text fill="#595959" font-family="Calibri,Calibri_MSFontService,sans-serif" font-weight="400" font-size="41" transform="matrix(1 0 0 1 2060.3 2891)">4</text></g><g clip-path="url(#clip16)"><text fill="#595959" font-family="Calibri,Calibri_MSFontService,sans-serif" font-weight="400" font-size="41" transform="matrix(1 0 0 1 2217.43 2891)">5 - Agree</text></g><g clip-path="url(#clip17)"><text fill="#595959" font-family="Calibri,Calibri_MSFontService,sans-serif" font-weight="400" font-size="46" transform="matrix(6.12323e-17 -1 1 6.12323e-17 1177.23 2868)">Number of Participants</text></g><g clip-path="url(#clip18)"><text fill="#595959" font-family="Calibri,Calibri_MSFontService,sans-serif" font-weight="400" font-size="46" transform="matrix(1 0 0 1 1666.88 2960)">Level of Agreement</text></g><g clip-path="url(#clip19)"><text fill="#595959" font-family="Calibri,Calibri_MSFontService,sans-serif" font-weight="400" font-size="64" transform="matrix(1 0 0 1 1317.42 2347)">Did the music fits to the situation?</text></g><rect x="1076.5" y="2252.5" width="1378" height="778" stroke="#D9D9D9" stroke-width="3.4375" stroke-linejoin="round" stroke-miterlimit="10" fill="none"/></g></svg>