Changes for page b. Test

Last modified by Demi Tao on 2023/04/10 10:13

From version 1.2
edited by Demi Tao
on 2023/03/23 09:22
Change comment: Added comment
To version 2.2
edited by Arber Demi
on 2023/03/25 17:05
Change comment: (Autosaved)

Summary

Details

Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.DemiTao
1 +XWiki.ArberDemi
Content
... ... @@ -2,27 +2,112 @@
2 2  
3 3  <include a short summary of the claims to be tested, i.e., the effects of the functions in a specfic use case>
4 4  
5 +<nothing
5 5  
7 +The main claims we are looking to test with this testing procedure are related to the functionality and usability of the robot.
8 +
9 +The participants will be other students taking the course. The participants will be placed in the shoes of a PwD and be tasked with completing several basic actions with the robot while impaired in several known ways to simulate the difficulties of a PwD.
10 +
11 +After the experiment, the participants will fill out a survey and be asked some more open ended questions with the purpose of understanding how the interaction with the robot went, and whether they have anything that they find concerning regarding the possible use of the system and its functions in a real life setting.
12 +
13 +On top of this, a short questionnaire will be sent to several care homes throughout the Netherlands in hopes to get a general idea whether the caretakers at the facilities think that the system would be a good fit for the proposed use case.
14 +
6 6  = 2. Method =
7 7  
17 +The prototypes are evaluated in a simulated manner, with participants pretending to be PwDs and conducting in-person experiments.
8 8  
9 9  == 2.1 Participants ==
10 10  
21 +All students in CS4235 Socio-Cognitive Engineering (2022-2023) in TU Delft are invited to test the robot. In the end, X students are presented.
11 11  
12 12  == 2.2 Experimental design ==
13 13  
25 +For the experiment, we used a within-subject design. All of the participants interacted with both versions of the robot, with half of the participants interacting with version 1 first and then version 2, and the other half in reverse order. This was done to counter-balance the carryover effects.
14 14  
15 15  == 2.3 Tasks ==
16 16  
29 +In the user test, the following tasks were asked of the participants:
17 17  
31 +==== Reminders for activities ====
32 +
33 +* (((
34 +Add a reminder that a relative will pay a visit tomorrow with the format as "sb. will come at 10 am on Friday". Set the reminder to remind you 10 min before that.
35 +)))
36 +* (((
37 +Add a reminder USING A VOICE COMMAND that today at 2 pm will have a general health checkup.
38 +)))
39 +* (((
40 +Check the reminders you have added for today and tomorrow.
41 +)))
42 +
43 +==== Personal profile ====
44 +
45 +* (((
46 +Add relatives as a contact in the "profile" section.
47 +)))
48 +
49 +==== Memory games ====
50 +
51 +* (((
52 +Go to the Games section and check what is included there.
53 +
54 +<this might not be part of the experiment as it might not get implemented in time)
55 +)))
56 +
57 +==== Medicine reminders ====
58 +
59 +(for professional caregivers, write it just in case)
60 +
61 +* (((
62 +In the section “My Health”, add a medicine reminder to take the medicine **Donepezil**, 1 time per day at 9 PM before going to bed.
63 +)))
64 +* (((
65 +Check medicines that have been added.
66 +)))
67 +* (((
68 +Delete medicines that have been added.
69 +)))
70 +
71 +==== About dementia ====
72 +
73 +* (((
74 +Go to the About dementia section and check the information provided.
75 +)))
76 +* (((
77 +Click on the different chapters and have a look at them.
78 +)))
79 +
80 +==== General tasks ====
81 +
82 +* (((
83 +Turn on the robot.
84 +)))
85 +
18 18  == 2.4 Measures ==
19 19  
88 +Quantitative measures are used in a user evaluation.  If a respondent had a minimum total score of 60% or more, he or she was considered to be satisfied with the application.
20 20  
21 21  == 2.5 Procedure ==
22 22  
92 +The procedure was conducted as follows:
23 23  
94 +1. Welcome participants and give an introduction.
95 +1. Get them to sign a consent form.
96 +1. Prepare them to pretend to be a person with dementia. *
97 +1. Have interaction with the robot and complete the tasks.
98 +1. Complete a questionnaire.
99 +1. Have a short interview with selected participants. (if possible, 2 participants)
100 +
101 +//* Several fingers taped together (to simulate PwD's inability to control movements flexibly);//
102 +
103 +// Wearing very dirty glasses (simulates blurred vision and degraded perception);//
104 +
105 +// Wearing headphones that broadcast murmurs (simulating hearing degradation and the noisy environment).//
106 +
24 24  == 2.6 Material ==
25 25  
109 +1. Consent form. To protect the privacy of participants and ensure the evaluation process goes smoothly, we will ask participants to sign a consent form, indicating they are willing to take part in the evaluation and the data gathered from the experiment will be analyzed by researchers.
110 +1. Pepper robot. <not sure how to elaborate on this>
26 26  
27 27  = 3. Results =
28 28  
XWiki.XWikiComments[1]
Author
... ... @@ -1,0 +1,1 @@
1 +Anonymous
Comment
... ... @@ -1,0 +1,40 @@
1 +Quintana et al. (2020) designed questions based on the most important quality attributes to evaluate how the application works for people with mild cognitive impairment.
2 +
3 +(potential questions tailored to our case)
4 +
5 +* (((
6 +How satisfied are you with the robot’s possibility to support you?
7 +)))
8 +* (((
9 +How well does the robot fulfill your expectations?
10 +)))
11 +* (((
12 +Imagine a perfect robot for this task. How far away from it is the robot you are using today?
13 +)))
14 +* (((
15 +I find the robot easily accessible for people with dementia.
16 +)))
17 +* (((
18 +I feel that I can trust the robot and that it is safe to use.
19 +)))
20 +* (((
21 +I find it easy to understand how to operate the robot.
22 +)))
23 +* (((
24 +I am able to understand all the information presented by the robot.
25 +)))
26 +* (((
27 +I feel that the robot gave me better control over my daily situation.
28 +)))
29 +
30 +The alternatives the users could give to all the above were the following: 1 = Strongly disagree; 2 = Disagree.; 3 = Neither agree nor disagree; 4 = Agree; 5 = Strongly agree.
31 +
32 +
33 +The System Usability Scale (SUS) is a dependable instrument for testing usability. It comprises of 10 questions with five response options which range from strongly agree to strongly disagree for responders (Jordan et al., 1996).
34 +
35 +(see the attachment for the 10 questions)
36 +
37 +
38 +1. Quintana M, Anderberg P, Sanmartin Berglund J, Frögren J, Cano N, Cellek S, Zhang J, Garolera M. Feasibility-Usability Study of a Tablet App Adapted Specifically for Persons with Cognitive Impairment—SMART4MD (Support Monitoring and Reminder Technology for Mild Dementia). //International Journal of Environmental Research and Public Health//. 2020; 17(18):6816. [[https:~~/~~/doi.org/10.3390/ijerph17186816>>https://doi.org/10.3390/ijerph17186816]]
39 +
40 +2. Brooke, J. SUS-A quick and dirty usability scale. //Usability Eval. Ind. **1996**, 189//, 4–7.
Date
... ... @@ -1,0 +1,1 @@
1 +2023-03-24 11:18:20.296