Skip to Content

Changes for page b. Test

Last modified by Ilinca Rentea on 2023/04/11 12:25
From version 6.1
edited by Varun Singh
on 2023/04/10 18:57
Change comment: There is no comment for this version
To version 3.1
edited by Varun Singh
on 2023/03/22 13:45
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -1,5 +1,9 @@
1 1  = 1. Introduction =
2 2  
3 +<include a short summary of the claims to be tested, i.e., the effects of the functions in a specific use case>
4 +
5 +
6 +(% style="line-height:1.38" %)
3 3  (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)The aim of our project was to provide support for people with dementia in their hobby of painting. This was done to bring some joy and comfort into their lives, especially if they previously enjoyed doing it. This could be done with a human caretaker but having a humanoid robot like Pepper has certain advantages in the overall painting activity.
4 4  
5 5  (% style="line-height:1.38" %)
... ... @@ -18,17 +18,17 @@
18 18  
19 19  == 2.2 Experimental Design ==
20 20  
21 -For the experiment, we used a within-subject design, as we did not have enough participants to do a statistically significant between-subject study. To mitigate the transfer effects caused by doing the same activity twice, we had half of the participants start with the control condition, and half of the participants start with the test condition.
25 +For the experiment, we used a between-subject design. Two control groups were set up and this allowed us to properly proceed ahead with the evaluation of our research question. The first control group did not interact with the robot at all whereas the second group did. This helped us to avoid bias within our results.
22 22  
23 23  == 2.3 Tasks ==
24 24  
25 -(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)We set up two tasks. One for the control condition and one for the test condition.
29 +(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)We set up two tasks. One for the control group and one for the test group.
26 26  
27 27  (% style="line-height:1.38" %)
28 -(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)For the test condition, we first provided the participant with painting and drawing utensils and a canvas. Then, we started up the robot to guide the activity. The robot was controlled through a Wizard of Oz system.
32 +(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)For the control group, we put each participant in front of a canvas and provided them with pencils and watercolors and asked them to paint whatever they wanted to.
29 29  
30 30  (% style="line-height:1.38" %)
31 -(% style="color:#000000; font-family:Arial; font-size:14.6667px; white-space:pre-wrap" %)For the control condition, we provided the participant with art supplies as well, but here the robot was not involved. Instead, they received an activity sheet which guided the participants in the same flow as the robot would do. The only main guidance that couldn't be given on paper was the search function for a suitable topic or personal picture to draw/paint.
35 +(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)For the test group, we provided them with the painting and drawing utensils and the canvas as well, but we had our robot guiding the activity. The robot was controlled through a Wizard of Oz system.
32 32  
33 33  == 2.4 Measures ==
34 34  
... ... @@ -35,13 +35,13 @@
35 35  (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)We solely used qualitative measures for our experiment, as painting is a highly subjective experience.
36 36  
37 37  (% style="line-height:1.38" %)
38 -(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)We gave each participant a questionnaire before and after the session.
42 +(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)We gave each participant a questionnaire before and after the session. In both the before and after questionnaires we measured their mood and satisfaction to derive the change that the session brought to their mental status.
39 39  
40 40  (% style="line-height:1.38" %)
41 -(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)In the before questionnaire, we asked them about their mood, painting experience and general attitude to robots. This was done to later check if those variables have interaction effects with the main results. See [[here >>https://forms.office.com/Pages/ResponsePage.aspx?id=TVJuCSlpMECM04q0LeCIewsut64TrQ5KmT3QxEFJ2z1URElCMFZGWFFOM085MFVMOExXNVdLRjRSNC4u]]the before questionnaire.
45 +(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)We also asked them for more specific feedback on different functions of the robot so we could find areas to improve, as this was the first prototype.
42 42  
43 43  (% style="line-height:1.38" %)
44 -(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)In the after questionnaire, we measured their experience of the activity, using an adapted version of the PACE questionnaire used in [1]. We also asked two extra questions to gauge their likelihood of long-term engagement and perceived self-competence. See [[here >>https://forms.office.com/Pages/ResponsePage.aspx?id=TVJuCSlpMECM04q0LeCIewsut64TrQ5KmT3QxEFJ2z1UN1M0T1pLU1hTN1NIRkw5MjhSWldPRzJUWC4u]]the after questionnaire.
48 +(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Lastly, we also measured their previous experience with painting, to see if the robot has a different impact on experienced and inexperienced painters.
45 45  
46 46  == 2.5 Procedure ==
47 47  
... ... @@ -50,10 +50,10 @@
50 50  
51 51  1. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Welcome participants and explain what they are going to be doing.
52 52  1. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Have them sign the consent form.
53 -1. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Complete Questionnaire 1 regarding:
57 +1. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Complete questionnaire 1 regarding:
54 54  11. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Their current mood
55 55  11. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Their previous painting/drawing experience
56 -1. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Have them do the painting session, either with the robot or not, depending on which group they belong to.
60 +1. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Have them do the painting session, either with the robot or not, depending which group they belonged to.
57 57  1. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Complete questionnaire 2 regarding:
58 58  11. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Their current mood
59 59  11. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Their satisfaction with the activity
... ... @@ -62,111 +62,15 @@
62 62  
63 63  == 2.6 Material ==
64 64  
65 -1. (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Consent form. To protect the privacy of participants and ensure that they accept that their data was analyzed by researchers, we had each participant fill in a consent form. This consent form and experiment were approved by the Ethics Commission of the TU Delft.
69 +1. (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Consent form. To protect the privacy of participants and ensure that they accept that their data was analyzed by researchers, we had each participant fill in a consent form. This consent form and experiment was approved by the Ethics Commission of the TU Delft.
66 66  1. (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Pepper robot. We programmed this robot to interact with the participants as described in our earlier sections.
67 -1. (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Utensils for painting and drawing: canvas, watercolours and pencils. We offer each participant the choice of whether they want to draw or paint.
71 +1. (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Utensils for painting and drawing: canvas, watercolors and pencils. We offer each participant the choice whether they want to draw or paint.
68 68  1. (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Microsoft forms. All the questionnaires were performed by Microsoft Forms, a GDPR/TU Delft approved tool for gathering data.
69 69  
70 70  = 3. Results =
71 71  
72 -(% class="wikigeneratedid" %)
73 -**RQ1: How does the assistance provided by a social robot influence engagement in art-related tasks for people with dementia?**
74 74  
75 -(% class="wikigeneratedid" %)
76 -
77 -○ Independent variable: with/without robot
78 -○ Dependent variable: level of engagement
79 -
80 -(% class="wikigeneratedid" %)
81 -
82 -**RQ2: How does the assistance provided by a social robot influence the level of agitation in art-related tasks for people with dementia?**
83 -
84 -(% class="wikigeneratedid" %)
85 -
86 -○ Independent variable: with/without robot
87 -○ Dependent variable: level of agitation
88 -
89 -
90 -(% class="wikigeneratedid" %)
91 -**PACES Questionnaire handpicked questions**
92 -
93 -(% class="wikigeneratedid" %)
94 -Q1 Did you enjoy the activity?
95 -
96 -(% class="wikigeneratedid" %)
97 -Q2 Did you feel interested in the activity?
98 -Q3 Did you like performing the activity?
99 -
100 -(% class="wikigeneratedid" %)
101 -Q4 Did you find the activity pleasurable?
102 -Q5 Were you absorbed in the activity?
103 -
104 -(% class="wikigeneratedid" %)
105 -Q6 Was the activity a lot of fun?
106 -
107 -(% class="wikigeneratedid" %)
108 -Q7 Did you find the activity energizing?
109 -
110 -(% class="wikigeneratedid" %)
111 -Q8 Did you the activity make you feel happy?
112 -
113 -(% class="wikigeneratedid" %)
114 -Q9 Was the activity pleasant?
115 -
116 -(% class="wikigeneratedid" %)
117 -Q10 Were you frustrated by the activity?
118 -
119 -(% class="wikigeneratedid" %)
120 -Q11 Did you find the activity to be gratifying?
121 -
122 -(% class="wikigeneratedid" %)
123 -Q12 Was the activity stimulating for you?
124 -
125 -(% class="wikigeneratedid" %)
126 -Q13 Did the activity give you a strong sense of accomplishment?
127 -Q14 Did you feel as though there is nothing else you would rather be doing?
128 -
129 -
130 -(% class="wikigeneratedid" %)
131 -**After Activity questions**
132 -
133 -(% class="wikigeneratedid" %)
134 -AQ1 I would like to paint more in the future with the robot.
135 -
136 -(% class="wikigeneratedid" %)
137 -AQ2 I feel more confident in painting by myself because of this activity.
138 -
139 -(% class="wikigeneratedid" id="HRQ1:" %)
140 -**Test Results:**
141 -
142 -The result of 1 is : MannwhitneyuResult(statistic=119.0, pvalue=0.16128927639663737)
143 -The result of 2 is : MannwhitneyuResult(statistic=124.5, pvalue=0.23160773305909133)
144 -The result of 3 is : MannwhitneyuResult(statistic=128.0, pvalue=0.2624600220829313)
145 -The result of 4 is : MannwhitneyuResult(statistic=170.0, pvalue=0.8068245764795088)
146 -The result of 5 is : MannwhitneyuResult(statistic=190.0, pvalue=0.37010115387134357)
147 -The result of 6 is : MannwhitneyuResult(statistic=154.5, pvalue=0.8038002705718079)
148 -The result of 7 is : MannwhitneyuResult(statistic=217.0, pvalue=0.07321208012798906)
149 -The result of 8 is : MannwhitneyuResult(statistic=134.0, pvalue=0.3680655063054846)
150 -The result of 9 is : MannwhitneyuResult(statistic=198.5, pvalue=0.23665845074130787)
151 -The result of 10 is : MannwhitneyuResult(statistic=191.0, pvalue=0.3521959055801669)
152 -The result of 11 is : MannwhitneyuResult(statistic=176.0, pvalue=0.6532884156524246)
153 -The result of 12 is : MannwhitneyuResult(statistic=141.5, pvalue=0.5153332738584296)
154 -The result of 13 is : MannwhitneyuResult(statistic=213.5, pvalue=0.09360342686552618)
155 -The result of 14 is : MannwhitneyuResult(statistic=91.0, pvalue=0.022196094647121984)
156 -
157 157  = 4. Discussion =
158 158  
159 -We followed a within-study approach with a control group and a test group. The control group was first instructed to perform the painting activity without Pepper and then move on to the same activity but this time with Pepper. The test group was told to do vice-versa to minimise any transfer effect.
160 160  
161 -We used questions Q2, Q3, Q5, Q13, Q14 to answer our research question 1 and questions Q1, Q4, Q6, Q7, Q8, Q9, Q10, Q11, Q12 to answer our research question 2. Based on the results we saw in the above sections, unfortunately, the results we obtained to answer our research questions related to Engagement and Agitation are not significant. These questions were picked from the PACES questionnaire (link). The aggregated score of the results based on the PACES questionnaire was 0.228 which is not significant enough for our threshold of 0.05. Although one interesting thing to note is that the raw scores for the questions were always higher for questions that were positively associated with the Pepper robot. Although this could also have been because the participants were our colleagues and they could have been biased in their responses.
162 -
163 -The participants were also required to answer two more questions AQ1 and AQ2 after the activity which was used to investigate whether the participants preferred the activity with a robot or without it. The mean responses for AQ1 were in favour of painting with the robot but for AQ2 no definite conclusion could be made. This could be because of the confounding factor related to the painting activity itself; participants who liked to paint preferred the activity either way with or without the robot.
164 -
165 -Overall our results are inconclusive in terms of statistical significance although the raw scores are in favour of the robot even with the limited participant size test that was conducted. Unfortunately based on the experiment we did, the limitations of the experiment such as not having actual PwDs we are unable to conclusively evaluate the added value of a robot.
166 -
167 167  = 5. Conclusions =
168 -
169 -
170 -=== References ===
171 -
172 -1. Mullen, S.P., Olson, E.A., Phillips, S.M. //et al.// Measuring enjoyment of physical activity in older adults: invariance of the physical activity enjoyment scale (paces) across groups and time. //Int J Behav Nutr Phys Act// **8**, 103 (2011). https:~/~/doi.org/10.1186/1479-5868-8-103