Skip to Content

b. Test

Version 3.2 by Marijn Roelvink on 2023/03/30 12:55

1. Introduction

<include a short summary of the claims to be tested, i.e., the effects of the functions in a specific use case>

The aim of our project was to provide support for people with dementia in their hobby of painting. This was done to bring some joy and comfort into their lives, especially if they previously enjoyed doing it. This could be done with a human caretaker but having a humanoid robot like Pepper has certain advantages in the overall painting activity. 

Pepper can improve and continue the engagement of PwDs towards painting. This would be done by gently triggering the PwD to paint and then providing support during the activity. Pepper will also as a result of the painting activity improve the quality and life of the PwD. This can be done by supporting them during painting and also playing some calming, ambient music. The painting activity would also bring together the PwD and their family since during the painting activity personalised recommendations could be given to the PwD to paint and once the painting is done, a picture could be taken and sent to the family. This should provide a much better connectedness and relatedness between the PwD and their family. Pepper would not do the painting activity for the PwD instead it will just guide them so it should also provide a much better sense of autonomy towards the task.

For all of these effects, we plan on using different questionnaires geared towards capturing the required measure to effectively evaluate our claims.

2. Method

The prototype was evaluated with an in-person experiment with multiple participants in a between-subject study.

2.1 Participants
Our colleagues from the course CS4235 SCE since the scope of the course does not allow us to evaluate our robot with actual PwDs.

2.2 Experimental Design

For the experiment, we used a within-subject design, as we did not have enough participants to do a statistically significant between-subject study. To mitigate the transfer effects caused by doing the same activity twice, we had half of the participants starting with the control condition, and half of the participants starting with the test condition. 

2.3 Tasks

We set up two tasks. One for the control condition and one for the test condition. 

For the test condition, we first provided the participant with painting and drawing utensils and a canvas. Then, we started up the robot to guide the activity. The robot was controlled through a Wizard of Oz system. 

For the control condition, we provided the participant with art supplies as well, but here the robot was not involved. Instead they received an activity sheet which guided the participants in the same flow as the robot would do. The only main guidance that couldn't be given on paper was the search function for a suitable topic or personal picture to draw/paint.

2.4 Measures

We solely used qualitative measures for our experiment, as painting is a highly subjective experience. 

We gave each participant a questionnaire before and after the session.

In the before questionnaire, we asked them about their mood, painting experience and general attitude to robots. This was done to later check if those variables have interaction effects with the main results. See here the before questionnaire.

In the after questionnaire, we measured their experience of the activity, using an adapted version of the PACE questionnaire used in [1]. We also asked two extra questions to gauge their likelihood of long-term engagement, and perceived self competence. See here the after questionnaire.

2.5 Procedure

The procedure was conducted as follows:

  1. Welcome participants and explain what they are going to be doing.
  2. Have them sign the consent form.
  3. Complete questionnaire 1 regarding:
    1. Their current mood
    2. Their previous painting/drawing experience
  4. Have them do the painting session, either with the robot or not, depending which group they belonged to. 
  5. Complete questionnaire 2 regarding:
    1. Their current mood
    2. Their satisfaction with the activity
    3. Whether they would like to paint more because of the activity
    4. Other feedback

2.6 Material

  1. Consent form. To protect the privacy of participants and ensure that they accept that their data was analyzed by researchers, we had each participant fill in a consent form. This consent form and experiment was approved by the Ethics Commission of the TU Delft.
  2. Pepper robot. We programmed this robot to interact with the participants as described in our earlier sections.
  3. Utensils for painting and drawing: canvas, watercolors and pencils. We offer each participant the choice whether they want to draw or paint. 
  4. Microsoft forms. All the questionnaires were performed by Microsoft Forms, a GDPR/TU Delft approved tool for gathering data. 

3. Results

RQ1:

The result of 1 is : Ttest_indResult(statistic=1.4975794013422261, pvalue=0.14346974449383315)
The result of 2 is : Ttest_indResult(statistic=-1.4223714338623947, pvalue=0.16402857963283687)
The result of 3 is : Ttest_indResult(statistic=-1.3247032534410947, pvalue=0.19410483688054006)
The result of 4 is : Ttest_indResult(statistic=0.21883172517636015, pvalue=0.8280894550130451)
The result of 5 is : Ttest_indResult(statistic=1.1103001118645561, pvalue=0.2746606492169669)
The result of 6 is : Ttest_indResult(statistic=-0.4963635881027171, pvalue=0.622831867082795)
The result of 7 is : Ttest_indResult(statistic=1.7856873313329575, pvalue=0.08307261813922004)
The result of 8 is : Ttest_indResult(statistic=-0.8367967692050108, pvalue=0.4085500396596581)
The result of 9 is : Ttest_indResult(statistic=0.9858982364095322, pvalue=0.3311474288136296)
The result of 10 is : Ttest_indResult(statistic=0.5978655614807643, pvalue=0.5538913868506702)
The result of 11 is : Ttest_indResult(statistic=0.6993338931663364, pvalue=0.48910044218053905)
The result of 12 is : Ttest_indResult(statistic=-0.6636373315087242, pvalue=0.5113976377267673)
The result of 13 is : Ttest_indResult(statistic=1.681821310426211, pvalue=0.10176776499485188)
The result of 14 is : Ttest_indResult(statistic=-2.6406213957953537, pvalue=0.012408357809218102)

4. Discussion

5. Conclusions

References

  1. Mullen, S.P., Olson, E.A., Phillips, S.M. et al. Measuring enjoyment of physical activity in older adults: invariance of the physical activity enjoyment scale (paces) across groups and time. Int J Behav Nutr Phys Act 8, 103 (2011). https://doi.org/10.1186/1479-5868-8-103