Changes for page Test

Last modified by Andrei Stefan on 2022/04/04 13:38

From version Icon 96.1 Icon
edited by Xinqi Li
on 2022/04/02 03:07
Change comment: There is no comment for this version
To version Icon 99.1 Icon
edited by Andrei Stefan
on 2022/04/04 12:08
Change comment: There is no comment for this version

Summary

Details

Icon Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.mona98
1 +XWiki.AndreiStefan
Content
... ... @@ -1,5 +1,7 @@
1 1  Our robot aims to help delay the stage of dementia or slow down the deterioration of memory. The best situation is that we can test the robot with real PwD and in a relatively long time period to see if this robot really works, which is impossible for our project. So our evaluation performs in a group control way. Participants are divided into two groups, group A with the intelligent one, and group B with the dumb one.
2 2  
3 +The differences between the dumb and intelligent robot are small. The latter tells users the right answer if they get it wrong, while the former only tells them that they have made a mistake.
4 +
3 3  = Problem statement and research questions =
4 4  
5 5  The main use cases that the evaluation focuses on are UC001: Daily todo list and UC005: Quiz. Based on the claims corresponding to those use cases, we derive the following research questions:
... ... @@ -27,7 +27,8 @@
27 27  == Experimental design ==
28 28  
29 29  The experiment will be conducted to simulate the reinforcement learning process of musical memory related to daily activities and to investigate if the quiz is indeed able to help with the learning.
30 -All participants would sign a consent form that informed them of the usage of the collected data and our goal of evaluations. In our prototype, users can personalize the association between music and activities based on their existing intrinsic knowledge. But due to the limited time and requiring a comparable result between groups, in evaluation, we forced 6 pieces of music and activities. Participants listened to the music and were asked the remember the associated activities.
32 +All participants would sign a consent form that informed them of the usage of the collected data and our goal of evaluations. In our prototype, users can personalize the association between music and activities based on their existing intrinsic knowledge. But due to the limited time and requiring a comparable result between groups, in evaluation, we forced 6 pieces of music and activities.
33 +Participants listened to the music and were asked the remember the associated activities. To this end, they were given a list with the 6 activities. In order to make it more difficult to remember (so they had to pretend less to have dementia), the music corresponding to the activities was played in a different order than that of the activities on the list. Furthermore, the music was quite similar (just instrumental). The users then had 3 minutes to practice with the NAO.
31 31  In the end, the participants would take a quiz to see how much they remembered. They are also asked to fill in a questionnaire including the feeling of the robot and possible feedback.
32 32  
33 33  1. How many questions did you answer correctly? (Points from 0-6)
... ... @@ -117,15 +117,11 @@
117 117  * As mentioned before, the small sample size made the accuracy of the result doubtable. Having a larger and more diverse sample group would allow us to more accurately predict real-world usage.
118 118  * The accuracy of the speech recognition system in the NAO and the availability of test subjects and robots also limited the evaluation.
119 119  
120 -In the future, we could improve in the following aspects,
123 +Based on our evaluation, we proved that participants with our intelligent robot are more willing to play the quiz and consider the robot can help them remember the task better compared with the control group. Our robot still needs further improvement based on the previous discussion. In the future, we could improve in the following aspects,
121 121  
122 122  * Test a full implementation of the system in a real setting with PwD.
123 123  * Research should also be done to look if the robot is actually necessary, or if the advantage of the system could be achieved by a cheaper alternative, such as a virtual robot on a tablet. (Also inspired by the feedback we got. One participant asked why we didn't create an APP.)
124 124  
125 -= Conclusion =
126 -
127 -Based on our evaluation, we proved that participants with our intelligent robot are more willing to play the quiz and consider the robot can help them remember the task better compared with the control group. Our robot still needs further improvement based on the previous discussion.
128 -
129 129  = Reference =
130 130  
131 -Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the system usability scale. Intl. Journal of Human–Computer Interaction, 24(6), 574-594.
130 +[1] Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the system usability scale. Intl. Journal of Human–Computer Interaction, 24(6), 574-594.