Skip to Content

Changes for page b. Test

Last modified by Demi Breen on 2023/04/09 15:10
From version 45.1
edited by Hugo van Dijk
on 2023/04/06 17:49
Change comment: There is no comment for this version
To version 51.1
edited by Maya Elasmar
on 2023/04/07 13:47
Change comment: There is no comment for this version

Summary

Details

Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.hjpvandijk
1 +XWiki.MayaElasmar
Content
... ... @@ -14,8 +14,6 @@
14 14  
15 15  - Whether there is a noticeable difference between emotion-based and goal-based; The PwD can communicate how he/she feels and score the walk.
16 16  
17 -- The robot is trustworthy and can be given the responsibility of going out with the PwD on its own
18 -
19 19  = 2. Method =
20 20  
21 21  Sadly enough we are not able to experiment on PwD in a real-time situation and over a longer time. The experiment will take place on one single day and thus the long-term effect of either goal-based or emotion-based motivation cannot be seen. However the difference in motivation can still five different results in a single experiment and these results can already show some promising results for further research.
... ... @@ -30,7 +30,7 @@
30 30  
31 31  **Methodological set-up:**
32 32  
33 -Pepper will be turned on and will start a conversation with the participant. It will ask the participant to go on a walk and based on the answer Pepper will go on the walk immediately or will try to motivate the PwD to go on a walk with him. During the walk, Pepper will ask the participant questions to keep the participant engaged and keep continuing on the walk. So for the experiment, a Pepper robot, the Choregraphe software and also freedom of movement are needed. The full step-by-step schedule of the experiment is given in the attachments for both the [[goal-based motivation>>attach:goal-based motivation.pdf]] and the [[emotion-based motivation>>attach:emotion-based motivation.pdf]].
31 +Pepper will be turned on and will start a conversation with the participant. It will ask the participant to go on a walk and based on the answer Pepper will go on the walk immediately or will try to motivate the PwD to go on a walk with him. During the walk, Pepper will ask the participant questions to keep the participant engaged and keep continuing on the walk. So for the experiment, a Pepper robot, the Choregraphe software and also freedom of movement are needed. The full step-by-step schedule of the experiment is given in the attachments for both the [[goal-based motivation>>attach:Ontology & robot design - goal-based.pdf]] and the [[emotion-based motivation>>attach:Ontology & robot design - emotion-based.pdf]].
34 34  
35 35  **Conditions**:
36 36  
... ... @@ -46,13 +46,15 @@
46 46  
47 47  The participant is expected to experiment according to the following plan:
48 48  
49 -* Step 1: The participant needs to stand close enough to the robot to have an interaction/conversation with Pepper
50 -* Step 2: The participant needs to look the robot in the eye
51 -* Step 3: The participant needs to actively answer the questions provided by Pepper during the experiment
52 -* Step 4: When the participant has been motivated to go on a walk he/she needs to walk for 1 to 2 meters with the robot
53 -* Step 5: During the walk, the participant is expected to contribute in some small talk with Pepper
54 -* Step 6: When the walk is finished the participant needs to communicate with Pepper after which Pepper will say goodbye and the actual experiment is over
55 -* Step 7: After the experiment, the participant needs to fill in the questionnaire provided by group 4
47 +* Step 1: The participant needs to fill in the consent form
48 +* Step 2: The participant needs to read the context information en emphasise as Bob
49 +* Step 3:The participant needs to stand close enough to the robot to have an interaction/conversation with Pepper
50 +* Step 4: The participant needs to look the robot in the eye
51 +* Step 5: The participant needs to actively answer the questions provided by Pepper during the experiment
52 +* Step 6: When the participant has been motivated to go on a walk he/she needs to walk for 1 to 2 meters with the robot
53 +* Step 7: During the walk, the participant is expected to contribute in some small talk with Pepper
54 +* Step 8: When the walk is finished the participant needs to communicate with Pepper after which Pepper will say goodbye and the actual experiment is over
55 +* Step 9: After the experiment, the participant needs to fill in the questionnaire provided by Group 4
56 56  
57 57  == 2.4 Measures ==
58 58  
... ... @@ -60,7 +60,7 @@
60 60  
61 61  In an optimal scenario where we can test the robot on PwD. We would have measured the number of times a person went out. We would also have measured the effect of the goal and emotion-based motivation on the long-term over the people. Whether it will be less effective over time or not. We would also measure the emotional effect on the caregivers and the functional effect. By the functional effect, we mean whether they indeed have more time to do other tasks or not. It would also have been perfect if we could measure the effect of the walks on the PwD and their health.
62 62  
63 -The questionnaire for the feedback is in the attachment (Questionnaire (2)). The questionnaire is based on a questionnaire in the paper " Measuring acceptance of an assistive social robot: a suggested toolkit " [2]. There are also 5 question at the end that we added ourselves, because we think it fits our experiment.
63 +The questionnaire for the feedback is in the attachment (Questionnaire (2)). The questionnaire is based on a questionnaire in the paper " Measuring acceptance of an assistive social robot: a suggested toolkit " [2]. There are also 5 questions at the end that we added ourselves because we think it fits our experiment.
64 64  
65 65  The questionnaire measures the experiment of the interaction of the students with the robot. By that we mean it measures:
66 66  
... ... @@ -74,23 +74,17 @@
74 74  
75 75   - the trustiness of the robot
76 76  
77 - -  the convenience of the reasons provided by the robots.
77 + - the convenience of the reasons provided by the robots.
78 78  
79 79  
80 80  == 2.5 Procedure ==
81 81  
82 +The procedure is as follows: we want the to test the claims mentioned above in the introduction. Therefore we programmed two routes in Choreography: one for the emotion-based motivation test and one for the goal-based motivation. To focus on only these two types of motivations, everything else in the route was kept the same. This is also to make sure that nothing else besides the motivation influences the participant's opinion on taking the walk.
82 82  
83 -The claims that need to be tested are thus:
84 +For the exeperiment we wrote an [[orientation script>>doc:3\. Evaluation.Scenario description.WebHome]] for the participants to introduce them to our design and explain them what they should do and that they should step into the shoes of our persona Bob. Bob is a person with anger issues and dementia. However, the participants did not know whether they are tested with emotion-based or goal-based motivation walk. We also wrote a consent form to ask for their consent to take part in the experiment. One of the main points in the consent from is that they will be recorded. We wanted to record them to re-evaluate all the experiments and see if we missed something. This also helped us with the final results and the discussion. If the participant did not agree, then we of course did not record him/her.
84 84  
85 -- The effect of emotion-based motivation; The PwD can comprehend the emotion that is being conveyed and in that way is motivated to contribute to the activity of walking in the garden.
86 +The following happend during an experiment:
86 86  
87 -- The effect of goal-based motivation; The PwD can comprehend the goal and end-state of the promoted activity and in that way is motivated to contribute to the activity of walking in the garden
88 -
89 -- Whether there is a noticeable difference between emotion-based and goal-based; The PwD can communicate how he/she feels and score the walk.
90 -
91 -
92 -The robot and students need to perform the following tasks:
93 -
94 94  ~1. Pepper will be turned on and will scan/check his environment
95 95  
96 96  2. Pepper will look for a face and will turn to the person that he sees
... ... @@ -101,13 +101,13 @@
101 101  
102 102  5. Pepper will start motivating based on the answer that the student gives:
103 103  
104 - 5.1 When the student says yes, Pepper will start walking with the student and during the walk will have some small talk
105 -5.2 When the student says no, Pepper will start the motivational part of the experiment. For the first experiment Pepper will use emotion-based motivation and for the second experiment Pepper will use the goal-based motivation
106 -5.3 If the student then decides to say yes, then Pepper will start walking with the student and during the walk will have some small talk
107 -6. After the walk/activity is finished Pepper will thank the student and will state again how important it is to stay active
98 + 5a. When the student says yes, Pepper will start walking with the student and during the walk will have some small talk
99 + 5b. When the student says no, Pepper will start the motivational part of the experiment. For the first experiment Pepper will use emotion-based  motivation and for the second experiment Pepper will use the goal-based motivation
100 + 5c. If the student then decides to say yes, then Pepper will start walking with the student and during the walk will have some small talk
108 108  
109 -7. Then the student is asked to answer some questions to evaluate the experience.
102 +During the experiment, one of us wrote down observations of the experiment and another one recoreded the experiment if allowed. We had also prepared a questionnaire to measure our claims, which we talked about in details in the measures section. All the participants had to fill in these questionnaire after the experiment. We wanted to make sure that we had an equal amount of both types of tests to get an unbiased result. Hence, we finished the evaluation once we had a relatively good and equal amount of experiments.
110 110  
104 +
111 111  == 2.6 Material ==
112 112  
113 113  The material needed for this experiment is of course the Pepper robot. We also need a laptop to run the robot.
... ... @@ -143,7 +143,7 @@
143 143  
144 144  - Mention something about only one participant going into Bob's character fully? And that he mentioned that the "no" he was giving was more attention-seeking than a real no.
145 145  
146 -- Add that sometimes the robot cut participants off, if they were speaking slower or elaborating on their answer.
140 +- Add that sometimes the robot cut participants off, if they were speaking slower or elaborating on their answers.
147 147  
148 148  
149 149  
... ... @@ -151,23 +151,23 @@
151 151  
152 152  - In terms of the research question, no significant differences were found. It could be that this is true in general, but it is very likely that this is influenced by the circumstances surrounding the design and the evaluation.
153 153  
154 -- The design is rather limited and with limited capabilities, due to time constraints. Speech recognition didn't always work properly and were not as flexible as desired which makes the interactions less realistic for the participant.
148 +- The design is rather limited and with limited capabilities, due to time constraints. Speech recognition didn't always work properly and was not as flexible as desired which made the interactions less realistic for the participant.
155 155  
156 -- There are also other constraints to the interaction, which has to be given as instructions to the participant before testing, such as at what distance to stay from the robot, when to join the robots side, how long to wait to speak after a certain prompt, etc. This further made it unnatural, but was necessary for the system to perform properly.
150 +- There are also other constraints to the interaction, which have to be given as instructions to the participant before testing, such as at what distance to stay from the robot, when to join the robot's side, how long to wait to speak after a certain prompt, etc. This further made it unnatural but was necessary for the system to perform properly.
157 157  
158 158  - Since participants were also prompted to give shorter answers and try to keep to things like "yes" and "no" it greatly influenced the way participants interacted with the robot.
159 159  
160 160  - Further, it was (obviously) not possible to test the design with PwD. This was attempted to be resolved by providing a persona description for participants to keep in mind during the testing. Only one participant ended up embodying this character.
161 161  
162 -- Results may have been different if participants outside of the course were used, since we are all very familiar with these robots and systems. On one hand it could be positive, since we have all researched dementia and have gained a lot of knowledge within this we could be better at simulating appropriate behavior with the robot or testing the systems in a reasonable way. But since participants also have an idea of how the robot works perhaps some mistakes or issues went undetected which could have appeared with individuals that are not familiar with the robot. Of course knowing about dementia is not the same thing as actually suffering from the diagnosis, so many aspects have most likely gone undetected there.
156 +- Results may have been different if participants outside of the course were used since we are all very familiar with these robots and systems. On one hand it could be positive, since we have all researched dementia and have gained a lot of knowledge within this we could be better at simulating appropriate behaviour with the robot or testing the systems in a reasonable way. But since participants also have an idea of how the robot works perhaps some mistakes or issues went undetected which could have appeared with individuals that are not familiar with the robot. Of course, knowing about dementia is not the same thing as actually suffering from the diagnosis, so many aspects have most likely gone undetected there.
163 163  
164 -- Results could also be influences by the sheer amount of participants, which concluded at 8 participants per group (8 for the goal-oriented approach, and 8 for the emotional approach). Perhaps with more participants the results would differ to a greater extent between the two approaches. Due to time constraints it was not possible to include more participants.
158 +- Results could also be influenced by the sheer amount of participants, which concluded at 8 participants per group (8 for the goal-oriented approach, and 8 for the emotional approach). Perhaps with more participants, the results would differ to a greater extent between the two approaches. Due to time constraints, it was not possible to include more participants.
165 165  
166 166  - Further participants who started the interaction with a pre-disposed idea of what they wanted to do, like the participant mentioned above in the results section, definitely influenced the outcome, since this was no longer about listening to the prompts the robot was giving.
167 167  
168 -- Interesting to consider is if participants are perhaps inclined to be positive, or feel like they need to be in such a project evaluation and if ideas like these also ended up affecting the outcome. ?
162 +- Interesting to consider if participants are perhaps inclined to be positive, or feel like they need to be in such a project evaluation and if ideas like these also ended up affecting the outcome. ?
169 169  
170 -- Normally, a robot should really take a walk outside. It should have been tested how a robot will do in actual garden, totally another surface then the room we did the experiment. Unfortunately, we could not do that, because we are not allowed to move th robot from the room.
164 +- Normally, a robot should really take a walk outside. It should have been tested how a robot will do in an actual garden, totally another surface then the room we did the experiment. Unfortunately, we could not do that, because we are not allowed to move th robot from the room.
171 171  
172 172  - In future studies the amount of participants should be considered, as well as testing the design on PwD and in a garden. Further improvements to the speech recognition are needed, as well as the smoothness of the walking and the distances travelled and the aspect of the participant's distance to the robot. Perhaps if the less realistic aspects discussed above are minimized, a robot that feels more realistic would result in participants listening to the actual prompts given, rather than going into the experiment with a predisposed idea of what they are going to do or answer and would also perhaps deter the participants from tending to reply positively.
173 173  
Ontology & robot design - emotion-based.pdf
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.Demibreen1000
Size
... ... @@ -1,0 +1,1 @@
1 +170.9 KB
Content
Ontology & robot design - goal-based.pdf
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.Demibreen1000
Size
... ... @@ -1,0 +1,1 @@
1 +191.9 KB
Content