Skip to Content

Changes for page b. Test

Last modified by Ilinca Rentea on 2023/04/11 12:25
From version 10.1
edited by Marijn Roelvink
on 2023/04/10 21:47
Change comment: There is no comment for this version
To version 3.1
edited by Varun Singh
on 2023/03/22 13:45
Change comment: There is no comment for this version

Summary

Details

Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.MarijnRoelvink
1 +XWiki.varunsingh3000
Content
... ... @@ -1,5 +1,9 @@
1 1  = 1. Introduction =
2 2  
3 +<include a short summary of the claims to be tested, i.e., the effects of the functions in a specific use case>
4 +
5 +
6 +(% style="line-height:1.38" %)
3 3  (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)The aim of our project was to provide support for people with dementia in their hobby of painting. This was done to bring some joy and comfort into their lives, especially if they previously enjoyed doing it. This could be done with a human caretaker but having a humanoid robot like Pepper has certain advantages in the overall painting activity.
4 4  
5 5  (% style="line-height:1.38" %)
... ... @@ -18,17 +18,17 @@
18 18  
19 19  == 2.2 Experimental Design ==
20 20  
21 -For the experiment, we used a within-subject design, as we did not have enough participants to do a statistically significant between-subject study. To mitigate the transfer effects caused by doing the same activity twice, we had half of the participants start with the control condition, and half of the participants start with the test condition.
25 +For the experiment, we used a between-subject design. Two control groups were set up and this allowed us to properly proceed ahead with the evaluation of our research question. The first control group did not interact with the robot at all whereas the second group did. This helped us to avoid bias within our results.
22 22  
23 23  == 2.3 Tasks ==
24 24  
25 -(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)We set up two tasks. One for the control condition and one for the test condition.
29 +(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)We set up two tasks. One for the control group and one for the test group.
26 26  
27 27  (% style="line-height:1.38" %)
28 -(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)For the test condition, we first provided the participant with painting and drawing utensils and a canvas. Then, we started up the robot to guide the activity. The robot was controlled through a Wizard of Oz system.
32 +(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)For the control group, we put each participant in front of a canvas and provided them with pencils and watercolors and asked them to paint whatever they wanted to.
29 29  
30 30  (% style="line-height:1.38" %)
31 -(% style="color:#000000; font-family:Arial; font-size:14.6667px; white-space:pre-wrap" %)For the control condition, we provided the participant with art supplies as well, but here the robot was not involved. Instead, they received an activity sheet which guided the participants in the same flow as the robot would do. The only main guidance that couldn't be given on paper was the search function for a suitable topic or personal picture to draw/paint.
35 +(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)For the test group, we provided them with the painting and drawing utensils and the canvas as well, but we had our robot guiding the activity. The robot was controlled through a Wizard of Oz system.
32 32  
33 33  == 2.4 Measures ==
34 34  
... ... @@ -35,13 +35,13 @@
35 35  (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)We solely used qualitative measures for our experiment, as painting is a highly subjective experience.
36 36  
37 37  (% style="line-height:1.38" %)
38 -(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)We gave each participant a questionnaire before and after the session.
42 +(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)We gave each participant a questionnaire before and after the session. In both the before and after questionnaires we measured their mood and satisfaction to derive the change that the session brought to their mental status.
39 39  
40 40  (% style="line-height:1.38" %)
41 -(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)In the before questionnaire, we asked them about their mood, painting experience and general attitude to robots. This was done to later check if those variables have interaction effects with the main results. See [[here >>https://forms.office.com/Pages/ResponsePage.aspx?id=TVJuCSlpMECM04q0LeCIewsut64TrQ5KmT3QxEFJ2z1URElCMFZGWFFOM085MFVMOExXNVdLRjRSNC4u]]the before questionnaire.
45 +(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)We also asked them for more specific feedback on different functions of the robot so we could find areas to improve, as this was the first prototype.
42 42  
43 43  (% style="line-height:1.38" %)
44 -(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)In the after questionnaire, we measured their experience of the activity, using an adapted version of the PACE questionnaire used in [1]. We also asked two extra questions to gauge their likelihood of long-term engagement and perceived self-competence. See [[here >>https://forms.office.com/Pages/ResponsePage.aspx?id=TVJuCSlpMECM04q0LeCIewsut64TrQ5KmT3QxEFJ2z1UN1M0T1pLU1hTN1NIRkw5MjhSWldPRzJUWC4u]]the after questionnaire.
48 +(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Lastly, we also measured their previous experience with painting, to see if the robot has a different impact on experienced and inexperienced painters.
45 45  
46 46  == 2.5 Procedure ==
47 47  
... ... @@ -50,10 +50,10 @@
50 50  
51 51  1. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Welcome participants and explain what they are going to be doing.
52 52  1. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Have them sign the consent form.
53 -1. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Complete Questionnaire 1 regarding:
57 +1. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Complete questionnaire 1 regarding:
54 54  11. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Their current mood
55 55  11. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Their previous painting/drawing experience
56 -1. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Have them do the painting session, either with the robot or not, depending on which group they belong to.
60 +1. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Have them do the painting session, either with the robot or not, depending which group they belonged to.
57 57  1. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Complete questionnaire 2 regarding:
58 58  11. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Their current mood
59 59  11. (% style="color:#333333; font-family:Arial; font-size:10.5pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Their satisfaction with the activity
... ... @@ -62,65 +62,15 @@
62 62  
63 63  == 2.6 Material ==
64 64  
65 -1. (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Consent form. To protect the privacy of participants and ensure that they accept that their data was analyzed by researchers, we had each participant fill in a consent form. This consent form and experiment were approved by the Ethics Commission of the TU Delft.
69 +1. (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Consent form. To protect the privacy of participants and ensure that they accept that their data was analyzed by researchers, we had each participant fill in a consent form. This consent form and experiment was approved by the Ethics Commission of the TU Delft.
66 66  1. (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Pepper robot. We programmed this robot to interact with the participants as described in our earlier sections.
67 -1. (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Utensils for painting and drawing: canvas, watercolours and pencils. We offer each participant the choice of whether they want to draw or paint.
71 +1. (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Utensils for painting and drawing: canvas, watercolors and pencils. We offer each participant the choice whether they want to draw or paint.
68 68  1. (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)Microsoft forms. All the questionnaires were performed by Microsoft Forms, a GDPR/TU Delft approved tool for gathering data.
69 69  
70 70  = 3. Results =
71 71  
72 -We grouped the results of the PACE questionnaires into two partitions:  One set of questions relating more to RQ1 and one set of questions relating more to RQ2.
73 73  
74 -
75 -(% class="wikigeneratedid" %)
76 -**RQ1: How does the assistance provided by a social robot influence engagement in art-related tasks for people with dementia?**
77 -
78 -[[image:RQ1.png||height="297" width="812"]]
79 -
80 -As one can see, the results were not exactly significant. The only question that obtained a reasonable p-score (P=0.022) was Q14. This implies that the robot did provide for a more immersive activity where the urge to do something else was diminished. Due to the fact that the robot gives more active guidance than paper instructions, it could be hypothesized that the participants were more actively feeling part of the activity.
81 -
82 -
83 -**RQ2: How does the assistance provided by a social robot influence the level of agitation in art-related tasks for people with dementia?**
84 -
85 -[[image:1681152536590-277.png||height="327" width="783"]]
86 -
87 -(% class="wikigeneratedid" %)
88 -For this research question the results also proved not to be conclusive. The only result that could be viewed as remotely significant is the result for question 7 (P=0.073). This could be caused either by the positive encouragement given by the robot, and/or the added functionality of searching for pictures to draw.
89 -
90 -(% class="wikigeneratedid" %)
91 -**Aggregated score**
92 -
93 -
94 -[[image:https://lh5.googleusercontent.com/8pM8mwBwL6UidLnOFDJ_Bjl6EA10teBaiLZe7Wseh8RXDvNImY4MiCzu-ygAUvMvUwBxlN5wV7hsHJZJHZ-x_yGbepWHWqfkLU9HL9mCyHwie8KrSlE9YVBSQuy2DaxRsLuIpRAbEISTCe4X0EFUVJOyuw=s2048||height="260" width="308"]]
95 -
96 -To assess the general positive impact of the robot, we also aggregated the scores to compared both test settings. This result also proved to be non-conclusive (P= 0.228).
97 -
98 -(% class="wikigeneratedid" %)
99 -**After Activity questions**
100 -
101 -(% class="wikigeneratedid" %)
102 -We also analysed the After Activity questions. These were asked to assess the robots' positive influence on [[EF01: Continued and improved engagement in painting>>doc:2\. Specification.Claims.E1.WebHome]] (Q3) and the robots' positive influence on the competence part of the Self Determination Theory (Q4).
103 -
104 -(% class="wikigeneratedid" %)
105 -[[image:1681154434106-542.png||height="249" width="582"]]
106 -
107 -(% class="wikigeneratedid" %)
108 -The students were agreeing on average with the statement "I would like to paint more in the future with the robot". However, for question 4, the general response was ambivalent. This is understandable as confidence in certain activity often only comes after multiple repetitions and not in one encounter.
109 -
110 110  = 4. Discussion =
111 111  
112 -The results do not show a conclusive effect in general towards the added value of having a robot performing the activity. This is can be attributed to different possible causes.
113 113  
114 -First of all, the questions that were asked were more focused on the experience of the activity rather than the specific added value a robot might give when doing such an activity. It is understandable that painting can be generally viewed as an enjoyable and engaging activity, so when focusing on the enjoyment, it could be understood that the robot does not make a significant impact there. However, the robot might give significant improvement in terms of support, motivation and structure during the activity. Especially if the test was conducted on actual PwDs. For future work it would be interesting to explore during the design process on which parts of the activity the robot might be able to do more than paper instructions and to investigate how these contributions might be measured.
115 -
116 -Moreover, as we alluded to earlier, the activity might be experienced highly different for PwDs than 20 year old TU Delft students. Therefore, it is hard to draw any conclusions on the use of the robot by testing it on people who do not need much support from it.
117 -
118 118  = 5. Conclusions =
119 -
120 -Our project sought to offer support to individuals suffering from dementia in their passion for painting, with the objective of enhancing their emotional well-being. While human caregivers could provide such assistance, utilizing a humanoid robot such as Pepper for painting activities provides several advantages. Pepper has the potential to enhance and sustain the interest of individuals with dementia in painting by encouraging them to paint and providing assistance during the activity. As a result, the quality of life of these individuals could be improved, which may be augmented by playing calming music during the painting process. In addition, this activity could bring together the individuals with their family members, as personalized painting recommendations could be provided, and photos of completed paintings could be shared. Rather than completing the painting activity for the individuals, Pepper would guide them through the process, promoting a greater sense of independence and self-sufficiency.
121 -
122 -We did a study to evaluate the effectiveness of having a robot like Pepper assisting the PwD with painting instead of the PwD doing the activity alone. Our results based on the responses obtained from our custom PACES questionnaire were statistically insignificant and hence we could not conclusively answer our research questions related to engagement and agitation. However, the raw scores of the questions were always in favour of having the robot while performing the activity which could indicate a positive attitude towards having the robot. Due to certain practical limitations of the course, we could not conduct the experiment with actual PwD or set it up in such a way but given enough time and effort the current implementation in Pepper could be extended to an actual care home with PwDs.
123 -
124 -=== References ===
125 -
126 -1. Mullen, S.P., Olson, E.A., Phillips, S.M. //et al.// Measuring enjoyment of physical activity in older adults: invariance of the physical activity enjoyment scale (paces) across groups and time. //Int J Behav Nutr Phys Act// **8**, 103 (2011). https:~/~/doi.org/10.1186/1479-5868-8-103
1681152536590-277.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.MarijnRoelvink
Size
... ... @@ -1,1 +1,0 @@
1 -177.5 KB
Content
1681154434106-542.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.MarijnRoelvink
Size
... ... @@ -1,1 +1,0 @@
1 -36.0 KB
Content
RQ1.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.MarijnRoelvink
Size
... ... @@ -1,1 +1,0 @@
1 -106.4 KB
Content