1. Introduction

With the previously described prototype, we can begin testing. The focus of our test is on CL001: Regular physical activity improves PwD physical well-being and CL002: Dance session improves PwD mental well-being, which are the main claims of our primary use case UC02.0: Dancing Session, more specifically UC02.1: Dancing Session Start. Through a before-and-after testing format with a 5-point Likert scale questionnaire and HR measurements, we assess whether our claims hold.

2. Method

2.1 Participants

Throughout TUDelft, 9 students (5 M, 4 F) were found who aided us in our experiments. We note these are not actual dementia patients, as we had limited access to actual dementia patients during the course of this experiment. While there were no inclusion or exclusion criteria, this puts our demographic around the age range of 18-27 (soft range), with participants having experience in some field in technology.

2.2 Experimental design

Our experiment made use of our prototype's functionality. A before-and-after test was performed to see if our dancing session validated CL002: Dance session improves PwD mental well-being by making them fill in an affect-focused questionnaire before and after the experience. Heart rate measurements were taken in the meantime to assess CL001: Regular physical activity improves PwD physical well-being. Further details are described in Measures.

2.3 Tasks

Experiment host: Inform the user of the experiment, ask the participant to fill in the consent form, ask the participant to fill in the questionnaires, time dancing session length.

Participant: Engage with the robot, wear fitbit device, dance, fill in 2 questionnaires.

Robot: Talk to the participant, encourage the user to dance, hold dance session.

2.4 Measures

We tested two claims, each through their own measures.

CL001: Regular physical activity improves PwD physical well-being: For this, we used a sports device similar to a fit-bit device, namely a Garmin Connect. After activating a workout session on the device, we quantitatively measure their heart rate and calories burned to measure their physical engagement. We also quantitatively measure the time spent dancing through a stopwatch, starting the first lap at the moment a song starts, and ending when the participant finishes the experiment. We expect to see heart rates entering the range of a light or medium intensity activity for our claim to hold.

CL002: Dance session improves PwD mental well-being: For this, we used a qualitative measure: We made a questionnaire featuring questions on a 5-point Likert scale (Strongly Disagree to Strongly Agree) to test one's current mental well-being. The results of the individual questionnaire rounds are averaged per question, allowing for a before-and-after comparison. The questions we asked were as follows, with inspiration taken from the PANAS questionnaire, originally devised by Watson et al. in 1988:

1. I feel motivated

2. I feel proud of myself

3. I am feeling irritable

4. I am feeling inspired

5. I feel miserable

6. I am content

7. I feel stressed

8. I am in a good mood

9. I feel physically exhausted

10. I would like to have a robot dog

11 (Round 2 only). Do you have any additional thoughts on the experience? 

For the above questions, the expectation is that values for our positive affect questions, 1, 2, 4, 6, 8, increase after the experiment. Meanwhile, our negative affect questions, 3, 5, 7, should decrease.

Two questions here are not focused on the claim itself. Question 9 is an additional question focused on confirming the physical exhaustion of our other claim, and is thereby expected to increase. Question 10 is an additional question that does not confirm a dedicated claim, but is there to provide insight into people's approval of Miro.

2.5 Procedure

1. We invite the participant to interact with our robot Miro-E. Before starting, they are asked to fill in the informed consent form.

2. If it is accepted, the user will wear the fit-bit device, and the procedure continues.

3. First, the user should fill in the first instance of the questionnaire.

4. The host of the experiment informs the user that they may start interacting with Miro-E at any time.

5. The user greets Miro-E. Here, we have the user inform Miro of their personal interests and proceed from UC03.1: Companion Mode.

6. After introducing each other, Miro-E encourages the user to start a "dancing session". 

7. The user requests to start the dance session.

8. Once done, we ask the participant to fill in the questionnaire once more.

2.6 Material

During the procedure we make use of three materials.

- A prototype mimicking Miro-E's functionality.

- A Garmin Connect device, which should be worn by the participant.

- A version of the Informed Consent Form, where it is acknowledged that the participant's answers to a questionnaire, as well as their metrics on a Garmin Connect device will be used for further analysis. The participant agrees that they are doing this of their own volition, and they may leave at any time. Only those participants who agreed to the terms engaged with the experiment. This consent form also highlights that this is focused on people with dementia.

3. Results

First, we go over CL002: Dance session improves PwD mental well-being. As can be seen in Figure 1, on average interacting with the prototype improved general mood for our participants.
Most of the positive emotions improve after interacting, while most of the negative emotions decrease in intensity after the interaction. This showcases the fact that the prototype mostly improves general mood and helps the participant feel better. It is interesting to note that the biggest gap occurs for Good mood and Stress. Interestingly, Physical exhaustion also decreases after dancing and interacting with the prototype, which we did not expect. The two feelings that worsened after the interaction are proudness and feeling miserable. 

Next, regarding CL001: Regular physical activity improves PwD physical well-being. Due to only having 1 Garmin Connect device, this experiment was performed on only half of the participants. By trimming the activities stored on the Garmin Connect device to the dancing session, measurements can be obtained for heart rates and calories burned while dancing, such as the following:

On average, we obtained the following results: The average activity heart rate was 103, with the max heart rate ranging from 124-136. This means participants tend to enter zone 2 for a portion of the duration, while usually being in zone 1. The total calories burned was 0-46, averaging at 21. The 0 calories burned was by a participant who did not dance during the entire duration of the dancing session, and instead sat still, listening to music. Finally, during the experiment, the avg music play length was 5:18. We note this was likely reduced due to students being busy.
 

4. Discussion

If these results had a large sample size, we could say that, given how most feelings positively changed, we believe that the results support our claim: Claim 02: PwD mental well-being improves. The two metrics that decrease are most likely related to the implementation of the interaction. Most participants complained that the voice recognition does not always work well, and that the conversation was very scripted from Miro. The frustration of being misheard paired with a scripted interaction most likely lead to our participants decrease in proudness and increase in feeling miserable. On the other hand, good music and dancing improved the general mood of participants.
Most of the participants did not go all out when dancing. Instead, they slightly moved around and danced without intensity. Based on the approximate calories burned and measured heart rate, they tend to enter Zone 2, often associated with light aerobic exercises. Participants most likely felt energized by the increase in heart rate without feeling the actual effort due to its light intensity, making the activity have the unintentional side-effect of working as a means to wake up properly.
Unfortunately, due to the small sample size and the Technical University background of our participants, it is unclear if these results are valid. More extensive testing on actual dementia patients would need to be performed to fully validate the above findings and our claims. Despite this, these results give early insights into potential benefits and limitations that our experiments contain.

5. Conclusions

Overall, the interaction supports our claim. However, there are still points that can be improved in the future.
Firstly, improving the flow of conversation and the voice recognition of Miro will most likely improve the two metrics that worsened during the experiment. As such, in a future iteration, the quality of voice recognition should become a strong new Requirement. Due to issues with background noise, we propose 'noise cancellation' to be included in this new Requirement.
Moreover, making Miro dance alongside the participant will also most likely help improve the mood by an even bigger margin. It would be interesting to test this with other robots as well. Dancing alongside a humanoid robot might make the participant more comfortable dancing, even in public spaces. To handle this in a future iteration, more extensive analysis on other Social Robots should be done, with the focus on making them compatible with our Use Cases by updating these. A Requirement on the dancing features of robots and their impact may also need to be included, alongside a tested claim to validate whether this impact is positive.
For testing, given the low sample size, testing with more participants will lead to more accurate results. While the current results prove the potential of this prototype, it can not be said with certainty that this is a good enough product for PwD.
Additionally, it may be of interest to try the experiment over an extended period of time, to confirm that the positive impact on participants does not stem from short-term novelty.