Prototype

In evaluation, we use the control group methods to evaluate our robot. For one group, participants interact with a dummy robot and another group interacts with the intelligent robot. The only difference between these two groups is the independent variable - dummy or intelligent robot, which makes our result more reasonable. Due to the instability of NAO's speech recognition, parts of the evaluation have to be hardcoded. Therefore our evaluation ended up as a "Wizard of Oz".

Difference of two groups

• Group A with the intelligent robot: The Intelligent robot would inform the participants of the correct answer when they gave a wrong one. This is simulating our use case 5, assuming the quiz can help PwDs enhance the memory between music and activities. 
• Group B with the dummy robot: The dummy robot can not correct the answer, which is trying to simulate the daily scenario that the robot only reminded the daily task but they had no idea whether PwD did the correct task and participants have no method to strengthen the intrinsic knowledge.

Evaluation Prototype

First, all participants would sign a consent form that informed them of the usage of the collected data and our goal of evaluations.
Step 1
In our prototype, users can personalize the link between music and activities based on their existing intrinsic knowledge. But due to the limited time and requiring a comparable result between groups, in evaluation, we forced 6 pieces of music and activities. Participants listened to the music and were asked the remember the associated activities.
Step 2
In the second step, participants in two groups would play with the robot for 3 minutes. 

1. The participant says "Quiz!"
   2. The robot says "Let's play the Quiz!", plays a random music piece, asks "what activity does this music relate to?" and waits for the response
   3. The participant answers,
   3.1 If the answer is correct, both the dummy and intelligent robot would reply "Yes, that's right!" and continue the next music.
   3.2 If the answer is wrong, the dummy robot would reply "That's not the activity this music relates to." and continue the next music. While the correct robot would reply "That's not the activity this music relates to. Please try again" and repeat the same music until it get the correct one.

Step 3
In the end, the participants would take a quiz to see how much they remembered. They are also asked to fill in a questionnaire including the feeling of the robot and possible feedback.