1. Introduction

For our research, we are looking into the effect of either using goal-based motivation or emotion-based motivation in promoting PwD for physical activity. Two systems will thus be designed; one motivating using emotion-based explanations and the other using goal-based motivation. The product will motivate the PwD to go for a walk in the park stimulating the amount of physical activity. It has been shown that physical activity, an increase in emotional stability and more goal-based activities can increase the mental and physical health of the PwD. Since 70% of the PwD have a lack of motivation, apathy and lack of interest in activities this project could have a great influence on the lives of these people. 

Thus our research question is:
What is the effect of goal-based and emotion-based explanations in prompting PwD for physical activity?
 

The claims that need to be tested are thus:

- The effect of emotion-based motivation; The PwD can comprehend the emotion that is being conveyed and in that way is motivated to contribute to the activity of walking in the garden. 

- The effect of goal-based motivation; The PwD can comprehend the goal and end-state of the promoted activity and in that way is motivated to contribute to the activity of walking in the garden

- Whether there is a noticeable difference between emotion-based and goal-based; The PwD can communicate how he/she feels and score the walk. 

2. Method

Sadly enough we are not able to experiment on PwD in a real-time situation and over a longer time. The experiment will take place on one single day and thus the long-term effect of either goal-based or emotion-based motivation cannot be seen. However the difference in motivation can still five different results in a single experiment and these results can already show some promising results for further research. 

2.1 Participants

The test will be with students from the University of Delft that are also following this course. For the results to be valid 15 participants in a research is the minimum amount as presented by researcher Marc Brysbaert [1]. Since the decision was made for in-between subject design, which will be elaborated more in the Experimental Design section, there are thus 30 participants necessary. Due to the time constraint and the number of students in the course, 30 participants will probably not be reachable for this experiment so fewer students will participate. Also, the use of a control group is not within reach because of the lack of resources. 

The participants are all young and do not have any form of dementia. The results of the research can thus not be seen as sufficient for a real-life implementation of the prototype. The students will be familiar with the robot and thus further explanation of the working of the robot is less relevant. Also, the students will feel more comfortable with the robot from the start. This might be very different to the PwD which might be a little hesitant in interacting with the robot. The integration of that aspect will be of great importance before real implementation. 

2.2 Experimental design

Methodological set-up:

Pepper will be turned on and will start a conversation with the participant. It will ask the participant to go on a walk and based on the answer Pepper will go on the walk immediately or will try to motivate the PwD to go on a walk with him. During the walk, Pepper will ask the participant questions to keep the participant engaged and keep continuing on the walk. So for the experiment, a Pepper robot, the Choregraphe software and also freedom of movement are needed. The full step-by-step schedule of the experiment is given in the attachments for both the goal-based motivation and the emotion-based motivation.

Conditions:

The interaction will take place in a TU Delft facility. The experiment will be held in the Insyght lab. Unfortunately, the space of the room is small compared to an actual garden. The robot needs some space to move, hence we will make sure to move everything moved out of the way and that other students (who are not experimenting at the moment) wait in the room next. Also, the walk will not be as long as it would be in the actual garden.

The room also has a different surface than an actual park. However, our experiment focused more on the motivation to go outside than the walk itself. The difference in surrounding, a room instead of a garden, might have a little effect on the experience of the participant. However, the motivation will probably not be affected by the surroundings as much. 

Subject design:

For the experiment it was chosen to do in-between subject design as the learning effect in the within-subject design might negatively impact the results. When the same participants have to perform the same routine twice, they might get bored and frustrated due to some repetitions in questioning in the different motivation methods. In the in-between-subject design, multiple experiments will be taken with different participants. The total group of participants will thus be divided in half regarding either the emotion-based or the goal-based motivation. Both are measured by a questionnaire, which will be discussed in more detail later on in the evaluation part.

2.3 Tasks

The participant is expected to experiment according to the following plan:

• Step 1: The participant needs to fill in the consent form
• Step 2: The participant needs to read the context information en emphasise as Bob
• Step 3:The participant needs to stand close enough to the robot to have an interaction/conversation with Pepper
• Step 4: The participant needs to look the robot in the eye 
• Step 5: The participant needs to actively answer the questions provided by Pepper during the experiment
• Step 6: When the participant has been motivated to go on a walk he/she needs to walk for 1 to 2 meters with the robot
• Step 7: During the walk, the participant is expected to contribute in some small talk with Pepper
• Step 8: When the walk is finished the participant needs to communicate with Pepper after which Pepper will say goodbye and the actual experiment is over
• Step 9: After the experiment, the participant needs to fill in the questionnaire provided by Group 4

2.4 Measures

If our situation, we would like to measure whether the provided motivation indeed affects the person. We would like to compare which one has more effect as well. That could be measured by seeing whether they indeed go on the walk or not. We would also like to measure how long it takes to convince them to go on a walk when they do not want. We would also like to measure their emotions during and after the walk. Did they enjoy it? Were they bored? Did they feel lonely? That could be measured with the feedback asked after the walk. 

In an optimal scenario where we can test the robot on PwD. We would have measured the number of times a person went out. We would also have measured the effect of the goal and emotion-based motivation on the long-term over the people. Whether it will be less effective over time or not. We would also measure the emotional effect on the caregivers and the functional effect. By the functional effect, we mean whether they indeed have more time to do other tasks or not. It would also have been perfect if we could measure the effect of the walks on the PwD and their health.

The questionnaire for the feedback is in the attachment (Questionnaire (2)). The questionnaire is based on a questionnaire in the paper " Measuring acceptance of an assistive social robot: a suggested toolkit " [2]. There are also 5 questions at the end that we added ourselves because we think it fits our experiment.

The questionnaire measures the experiment of the interaction of the students with the robot. By that we mean it measures:

 - the usability of the robot

 - the enjoyment of communicating with the robot

 - the usefulness of the robot

 - the humanization of the robot, how much does the robot feel like a real person

 - the trustiness of the robot

 - the convenience of the reasons provided by the robots.

2.5 Procedure

The procedure is as follows: we want the to test the claims mentioned above in the introduction. Therefore we programmed two routes in Choreography: one for the emotion-based motivation test and one for the goal-based motivation. To focus on only these two types of motivations, everything else in the route was kept the same. This is also to make sure that nothing else besides the motivation influences the participant's opinion on taking the walk. 

For the exeperiment we wrote an orientation script for the participants to introduce them to our design and explain them what they should do and that they should step into the shoes of our persona Bob. Bob is a person with anger issues and dementia. However, the participants did not know whether they are tested with emotion-based or goal-based motivation walk. We also wrote a consent form to ask for their consent to take part in the experiment. One of the main points in the consent from is that they will be recorded. We wanted to record them to re-evaluate all the experiments and see if we missed something. This also helped us with the final results and the discussion. If the participant did not agree, then we of course did not record him/her.

The following happend during an experiment:

1. Pepper will be turned on and will scan/check his environment 

2. Pepper will look for a face and will turn to the person that he sees

3. Pepper will recognize a face and will introduce itself to the student

4. Pepper will then ask to go for a walk after which the student can either say yes or no

5. Pepper will start motivating based on the answer that the student gives:

         5a. When the student says yes, Pepper will start walking with the student and during the walk will have some small talk
         5b. When the student says no, Pepper will start the motivational part of the experiment. For the first experiment Pepper will use emotion-based                       motivation and for the second experiment Pepper will use the goal-based motivation
         5c. If the student then decides to say yes, then Pepper will start walking with the student and during the walk will have some small talk

During the experiment, one of us wrote down observations of the experiment and another one recoreded the experiment if allowed. We had also prepared a questionnaire to measure our claims, which we talked about in details in the measures section. All the participants had to fill in these questionnaire after the experiment. We wanted to make sure that we had an equal amount of both types of tests to get an unbiased result. Hence, we finished the evaluation once we had a relatively good and equal amount of experiments. 

2.6 Material

The material needed for this experiment is of course the Pepper robot. We also need a laptop to run the robot.

3. Results

Noteworthy answers

On average, participants only rejected the robot's persuasion attempts 0.5 times. The participants rated the robot a 2/5 in terms of being scary. They gave a 4/5 for it making life more interesting and it being good to make use of the robot. Questions related to the participant's enjoyment and fascination with the system and the robot were met with ratings between 3.8 and 4.1. The question "I think the staff would like me using the robot" was rated a 4/5 on average. A 2.3/5 was given to the statement "The robot insisted too much to go on a walk". Finally, to the question of whether they would not have gone for a walk if the robot didn't ask them to, the average answer was 3.8/5. All these answers had a standard deviation of less than 1.

ANOVA

Firstly, the Jarque-Bera test [3] was used to check for normality. When the answers to a question weren't normally distributed, the Mann-Whitney U-Test [4] was used. For normally distributed answers, the T-Test [5] was used. These tests used the null hypothesis that there is no significant difference between the two groups. When the calculated probability value (p-value) is less than 0.05, we can reject the null hypothesis and conclude that there is a significant difference between the two groups for the answers to that question.

Even though the average rejections were higher for emotion-based (0,875) than for goal-based(0,125). This difference was not significant. 

Furthermore, there was no significant difference in any of the questionnaire answers between the two groups.

This table shows the p-value per measure.

Observations

General remarks made by participants evaluating the emotion-based system were only about the walking aspect of the robot, stating that the walking distance should be increased and the change in direction was quite sharp. Participants doing the goal-based evaluation commented on the badly performing speech recognition system and stated that it might be useful to start by asking how the participant feels.

When asked the reason that convinced the participant to join the robot on a walk, two out of the six participants that said yes eventually in the emotion-based system recited one of the persuasion subjects. For the goal-based system, this was three out of eight.

When participants were standing too close to the robot, it wouldn't walk. This happened numerous times, resulting in conversation without walking.

Even though it was specified at the start of every session that the participant can say either yes or no to the robot's persuasion attempts, we noticed that some participants did not seem to grasp the fact that they could say no. At the end of their session, one participant stated that he was not persuaded by the robot at all, even though they said yes on the robot's first persuasion attempt.

Another participant, who said no to all persuasion attempts, stated afterwards that they "Just wanted to see what would happen if I said no all the time".  This indicated that some participants already had a plan of how many times they would reject the robot before starting, and did not really listen to the persuasions made.

As the robot's speech recognition could only understand single words due to its implementation, this resulted in numerous occasions where a participant was not understood and had to repeat themselves. It also occurred that the robot understood 'yes' when 'no' was said. 

In total during all of the evaluations performed, only one participant went into the Bob persona fully, which was described for the participant in the orientation script. He mentioned that the "no" he was giving during the test was more attention-seeking than a real no to the walk, which is a very useful observation. 

In a couple of the evaluations, it happened that the robot cut participants off mid-sentence once it had recognized a word that was spoken if they were speaking slower or elaborating on their answers. This is not ideal for a future and complete design and definitely would be something that needs to be worked on. 

4. Discussion

- In terms of the research question, no significant differences were found. It could be that this is true in general, but it is very likely that this is influenced by the circumstances surrounding the design and the evaluation. 

- The design is rather limited and with limited capabilities, due to time constraints. Speech recognition didn't always work properly and was not as flexible as desired which made the interactions less realistic for the participant. 

- There are also other constraints to the interaction, which have to be given as instructions to the participant before testing, such as at what distance to stay from the robot, when to join the robot's side, how long to wait to speak after a certain prompt, etc. This further made it unnatural but was necessary for the system to perform properly. 

- Since participants were also prompted to give shorter answers and try to keep to things like "yes" and "no" it greatly influenced the way participants interacted with the robot. 

- Further, it was (obviously) not possible to test the design with PwD. This was attempted to be resolved by providing a persona description for participants to keep in mind during the testing. Only one participant ended up embodying this character. 

- Results may have been different if participants outside of the course were used since we are all very familiar with these robots and systems. On one hand it could be positive, since we have all researched dementia and have gained a lot of knowledge within this we could be better at simulating appropriate behaviour with the robot or testing the systems in a reasonable way. But since participants also have an idea of how the robot works perhaps some mistakes or issues went undetected which could have appeared with individuals that are not familiar with the robot. Of course, knowing about dementia is not the same thing as actually suffering from the diagnosis, so many aspects have most likely gone undetected there. 

- Results could also be influenced by the sheer amount of participants, which concluded at 8 participants per group (8 for the goal-oriented approach, and 8 for the emotional approach). Perhaps with more participants, the results would differ to a greater extent between the two approaches. Due to time constraints, it was not possible to include more participants. 

- Further participants who started the interaction with a pre-disposed idea of what they wanted to do, like the participant mentioned above in the results section, definitely influenced the outcome, since this was no longer about listening to the prompts the robot was giving. 

- Interesting to consider if participants are perhaps inclined to be positive, or feel like they need to be in such a project evaluation and if ideas like these also ended up affecting the outcome. ?

- Normally, a robot should really take a walk outside. It should have been tested how a robot will do in an actual garden, totally another surface then the room we did the experiment. Unfortunately, we could not do that, because we are not allowed to move th robot from the room.

- In future studies the amount of participants should be considered, as well as testing the design on PwD and in a garden. Further improvements to the speech recognition are needed, as well as the smoothness of the walking and the distances travelled and the aspect of the participant's distance to the robot. Perhaps if the less realistic aspects discussed above are minimized, a robot that feels more realistic would result in participants listening to the actual prompts given, rather than going into the experiment with a predisposed idea of what they are going to do or answer and would also perhaps deter the participants from tending to reply positively. 

ADD FUTURE WORK

5. Conclusions

Both systems were deemed enjoyable and fascinating, and little rejections were made to both types of persuasions. No significant difference was found in any of the measures between the two groups.

References

[1] Brysbaert, M. (2019). How many participants do we have to include in properly powered experiments? A tutorial of power analysis with reference tables. Journal of Cognition, 2(1), 16. DOI: http://doi.org/10.5334/joc.72

[2] M. Heerink, B. Kröse, V. Evers, and B. Wielinga, “Measuring acceptance of an assistive social robot: a suggested toolkit .” [Online]. Available: https://mheerink.home.xs4all.nl/pdf/HeerinkRo-man09.pdf.

[3] Thorsten Thadewald and Herbert Büning. “Jarque–Bera test and its competitors for testing
normality–a power comparison”. In: Journal of applied statistics 34.1 (2007), pp. 87–105.

[4] Nadim Nachar et al. “The Mann-Whitney U: A test for assessing whether two indepen-
dent samples come from the same distribution”. In: Tutorials in quantitative Methods for
Psychology 4.1 (2008), pp. 13–20.

[5] Tae Kyun Kim. “T test as a parametric statistic”. In: Korean journal of anesthesiology 68.6
(2015), pp. 540–546.