Skip to Content

Changes for page Test

Last modified by Andrei Stefan on 2022/04/04 13:38
From version Icon 48.1 Icon
edited by Dongxu Lu
on 2022/03/22 13:22
Change comment: There is no comment for this version
To version Icon 52.1 Icon
edited by Xinqi Li
on 2022/04/01 16:32
Change comment: There is no comment for this version

Summary

Details

Icon Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.dongxulu
1 +XWiki.mona98
Content
... ... @@ -1,3 +1,4 @@
1 +
1 1  = Problem statement and research questions =
2 2  
3 3  The main use cases that the evaluation focuses on are UC001: Daily todo list and UC005: Quiz. Based on the claims corresponding to those use cases, we derive the following research questions:
... ... @@ -11,7 +11,7 @@
11 11  
12 12  = Method =
13 13  
14 -We are doing a mixed-method approach. We are producing quantitative data from questionnaires after the interaction with the robot, by asking them how they felt about the whole experience. By measuring this data, we will assess if we successfully achieved our claims and determine the answers to the research questions.
15 +Our group decided to use a mixed-method approach for the evaluation. Quantitative data will be derived during the experiment such as the number of mistakes the participant makes during the quiz. Moreover, qualitative data expected to be gathered through questionnaires, such as to what extent participants are satisfied with using the robot, is also adopted for evaluation. By measuring these two types of data, we will manage to assess if our claims are achieved and the research questions are answered.
15 15  
16 16  == Participants ==
17 17  
... ... @@ -19,13 +19,13 @@
19 19  
20 20  == Experimental design ==
21 21  
22 -The experiment will be conducted by simulating the interaction between the patient, robot and other actors. The group of participants will be divided into 3 subgroups that simulate patients in different stages of dementia. The participants will be given artificial memory loss by forcing them to remember a large number of songs they have to associate with certain activities. From each group one of the participants will be asked to play the caregiver and another participant will take the roll of loved one. With these participants the interactions between the actors will be tested. The interactions are described in the design patterns.
23 +The experiment will be conducted to simulate the reinforcement learning process of musical memory related to daily activities and to investigate if the quiz is indeed able to help with the learning. The group of participants will be divided into 3 subgroups that simulate patients in different stages of dementia. The participants will be given artificial memory loss by forcing them to remember a large number of songs they have to associate with certain activities. From each group one of the participants will be asked to play the caregiver and another participant will take the roll of loved one. With these participants the interactions between the actors will be tested. The interactions are described in the design patterns.
23 23  
24 24  When the participant playing the role of patient has learned the association between the music pieces and activities the robot will start playing certain pieces of music. The participant has to recall the correct activity associated with the music piece. When this is wrong the loved one can step in and call to remind the
25 25  
26 26  == Tasks ==
27 27  
28 -**Event: Activity**
29 +**Event: Quiz**
29 29  
30 30  {{html}}
31 31  <table>
... ... @@ -33,84 +33,25 @@
33 33   <td>No.</td>
34 34   <td>Group A</td>
35 35   <td>Group B</td>
36 - <td>Group C</td>
37 37   </tr>
38 38   <tr>
39 39   <td>1</td>
40 - <td>Memorize five pieces of music corresponding with different activities within three minutes;</td>
41 - <td>Memorize seven pieces of music corresponding with different activities within three minutes;</td>
42 - <td>Memorize ten pieces of music corresponding with different activities within three minutes;</td>
40 + <td>Memorize six pieces of music corresponding with different activities;</td>
41 + <td>Memorize six pieces of music corresponding with different activities;</td>
43 43   </tr>
44 44   <tr>
45 45   <td>2</td>
46 - <td>Say “I will do [activity_name] now.”/ ”I don’t know.” to the robot after hearing the robot play music1;</td>
47 - <td>Say “I will do [activity_name] now.”/ ”I don’t know.” to the robot after hearing the robot play music1;</td>
48 - <td>Say “I will do [activity_name] now.”/ ”I don’t know.” to the robot after hearing the robot play music1;</td>
45 + <td>Play quiz with the dumb robot for five minutes, which will not correct the participant when wrong answers are given ;</td>
46 + <td>Play quiz with the smart robot for five minutes, which will correct the participant when wrong answers are given;</td>
49 49   </tr>
50 50   <tr>
51 51   <td>3</td>
52 - <td>Say “I will do [activity_name] now.”/ ”I don’t know.” to the robot after hearing the robot play music2;</td>
53 - <td>Say “I will do [activity_name] now.”/ ”I don’t know.” to the robot after hearing the robot play music2;</td>
54 - <td>Say “I will do [activity_name] now.”/ ”I don’t know.” to the robot after hearing the robot play music2;</td>
50 + <td>Test how well the participant remember the music-activity pairs by counting the mistakes made;</td>
51 + <td>Test how well the participant remember the music-activity pairs by counting the mistakes made;</td>
55 55   </tr>
56 - <tr>
57 - <td>4</td>
58 - <td>Ignore the music when hearing the robot playing music3;</td>
59 - <td>Ignore the music when hearing the robot playing music3;</td>
60 - <td>Ignore the music when hearing the robot playing music3;</td>
61 - </tr>
62 - <tr>
63 - <td>5</td>
64 - <td>Say “I will do [task_name] now.”/ ”I don’t know.” to the robot after hearing the robot play music3 again.</td>
65 - <td>Say “I will do [task_name] now.”/ ”I don’t know.” to the robot after hearing the robot play music3 again.</td>
66 - <td>Say “I will do [task_name] now.”/ ”I don’t know.” to the robot after hearing the robot play music3 again.</td>
67 - </tr>
68 68  <table>
69 69  {{/html}}
70 70  
71 -**Event: Quiz**
72 -
73 -{{html}}
74 -<table>
75 - <tr>
76 - <td>No.</td>
77 - <td>Group A</td>
78 - <td>Group B</td>
79 - <td>Group C</td>
80 - </tr>
81 - <tr>
82 - <td>1</td>
83 - <td>Say "I would like to do a quiz now." to the robot;</td>
84 - <td>Say "I would like to do a quiz now." to the robot;</td>
85 - <td>Say "I would like to do a quiz now." to the robot;</td>
86 - </tr>
87 - <tr>
88 - <td>2</td>
89 - <td>Say “The activity is [activity_name].”/ ”I don’t know.” to the robot after hearing the robot play music1;</td>
90 - <td>Say “The activity is [activity_name].”/ ”I don’t know.” to the robot after hearing the robot play music1;</td>
91 - <td>Say “The activity is [activity_name].”/ ”I don’t know.” to the robot after hearing the robot play music1;</td>
92 - </tr>
93 - <tr>
94 - <td>3</td>
95 - <td>Say “The activity is [activity_name].”/ ”I don’t know.” to the robot after hearing the robot play music2;</td>
96 - <td>Say “The activity is [activity_name].”/ ”I don’t know.” to the robot after hearing the robot play music2;</td>
97 - <td>Say “The activity is [activity_name].”/ ”I don’t know.” to the robot after hearing the robot play music2;</td>
98 - </tr>
99 - <tr>
100 - <td>4</td>
101 - <td>Say “The activity is [activity_name].”/ ”I don’t know.” to the robot after hearing the robot play music3;</td>
102 - <td>Say “The activity is [activity_name].”/ ”I don’t know.” to the robot after hearing the robot play music3;</td>
103 - <td>Say “The activity is [activity_name].”/ ”I don’t know.” to the robot after hearing the robot play music3;</td>
104 - </tr>
105 - <tr>
106 - <td>5</td>
107 - <td>Say “I want to end the quiz now.” to the robot.</td>
108 - <td>Say “I want to end the quiz now.” to the robot.</td>
109 - <td>Say “I want to end the quiz now.” to the robot.</td>
110 - </tr>
111 -<table>
112 -{{/html}}
113 -
114 114  == Measures ==
115 115  
116 116  During the experiment, count how many tries it takes the user to answer correctly