Skip to Content

Changes for page b. Test

Last modified by Demi Tao on 2023/04/10 10:13
From version 10.1
edited by Karthik Prakash
on 2023/04/07 19:19
Change comment: There is no comment for this version
To version 17.1
edited by Demi Tao
on 2023/04/08 01:14
Change comment: There is no comment for this version

Summary

Details

Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.KarthikPrakash
1 +XWiki.DemiTao
Content
... ... @@ -70,7 +70,7 @@
70 70  
71 71  Two quantitative measures were employed in the user evaluation. The first measure aimed to test attributes including accessibility, trustworthiness, and comprehensibility. The second measure used was the System Usability Scale (SUS), which is a widely-used scale for evaluating the usability of the software.
72 72  
73 -In addition to the quantitative measures, a structured interview with open-ended questions was conducted. The aim of this interview was to gain a deeper understanding of participants' experiences with the robot, including any concerns they may have regarding the potential use of the system and its functions in a real-life setting.
73 +In addition to the quantitative measures, a structured interview with open-ended questions was conducted with randomly selected participants. The aim of this interview was to gain a deeper understanding of participants' experiences with the robot, including any concerns they may have regarding the potential use of the system and its functions in a real-life setting.
74 74  
75 75  By using both quantitative and qualitative measures, the user evaluation can provide a more comprehensive understanding of participants' experiences and perceptions of the system. This approach allows for a more nuanced analysis of the data and can provide valuable insights into the strengths and weaknesses of the system, as well as areas for improvement.
76 76  
... ... @@ -93,16 +93,73 @@
93 93  
94 94  = 3. Results =
95 95  
96 -|=(% style="width: 199px;" %)Tasks|=(% style="width: 147px;" %)Succeded by Themselves|=(% style="width: 146px;" %)Succeded with Some Guidance|=(% style="width: 185px;" %)Succeded with Detailed Explicit Instructions|=(% style="width: 175px;" %)Average Time to Complete Task (s)
97 -|(% style="width:199px" %)Turning the robot on and off|(% style="width:147px" %) |(% style="width:146px" %) |(% style="width:185px" %) |(% style="width:175px" %)
98 -|(% style="width:199px" %)Add a reminder using "+" button|(% style="width:147px" %) |(% style="width:146px" %) |(% style="width:185px" %) |(% style="width:175px" %)
99 -|(% style="width:199px" %)Add a reminder using voice|(% style="width:147px" %) |(% style="width:146px" %) |(% style="width:185px" %) |(% style="width:175px" %)
100 -|(% style="width:199px" %)Check the reminder|(% style="width:147px" %) |(% style="width:146px" %) |(% style="width:185px" %) |(% style="width:175px" %)
101 -|(% style="width:199px" %)Create a personal profile|(% style="width:147px" %) |(% style="width:146px" %) |(% style="width:185px" %) |(% style="width:175px" %)
96 +=== (% style="color:inherit; font-family:inherit" %)Results of the survey:(%%) ===
102 102  
103 -if possible, note down some Parts Where Users Struggled in each task.
98 +[[image:attach:chart.png||alt="Survey Results"]]
104 104  
100 +|(% style="width:215px" %)**Attributes**|(% style="width:211px" %)**Mean (control group)**|(% style="width:229px" %)**Mean (Experimental group)**|(% style="width:197px" %)**P-value**
101 +|(% style="width:215px" %)Accessibility|(% style="width:211px" %)2,33|(% style="width:229px" %)3,25  |(% style="width:197px" %)0,0644
102 +|(% style="width:215px" %)Trustability|(% style="width:211px" %)3,83|(% style="width:229px" %)4,125|(% style="width:197px" %)0,3165
103 +|(% style="width:215px" %)Perceivability|(% style="width:211px" %)3,33|(% style="width:229px" %)3,5|(% style="width:197px" %)0,4112
104 +|(% style="width:215px" %)Understandability|(% style="width:211px" %)3,33|(% style="width:229px" %)4,25|(% style="width:197px" %)0,1151
105 +|(% style="width:215px" %)Empowerment|(% style="width:211px" %)3,33|(% style="width:229px" %)4|(% style="width:197px" %)0,0895
106 +|(% style="width:215px" %)Usability|(% style="width:211px" %)54,16666667|(% style="width:229px" %)71,875|(% style="width:197px" %)0,0903
107 +
108 +=== Observation: ===
109 +
110 +|=(% style="width: 199px;" %)Tasks|=(% style="width: 147px;" %)Succeeded by Themselves|=(% style="width: 146px;" %)Succeeded with Some Guidance|=(% style="width: 185px;" %)Succeeded with Detailed Explicit Instructions|=(% style="width: 175px;" %)Average Time to Complete Task (s)
111 +|(% style="width:199px" %)Add a reminder|(% style="width:147px" %)14.29%|(% style="width:146px" %)28.57%|(% style="width:185px" %)57.14%|(% style="width:175px" %)97
112 +|(% style="width:199px" %)Check weekly remainders on the Calendar page|(% style="width:147px" %)100%|(% style="width:146px" %)0%|(% style="width:185px" %)0%|(% style="width:175px" %)36
113 +|(% style="width:199px" %)Create a personal profile|(% style="width:147px" %)7.14%|(% style="width:146px" %)50%|(% style="width:185px" %)42.86%|(% style="width:175px" %)69
114 +|(% style="width:199px" %)Verify current profiles|(% style="width:147px" %)85.71%|(% style="width:146px" %)14.29%|(% style="width:185px" %)0%|(% style="width:175px" %)32
115 +|(% style="width:199px" %)Play memory game|(% style="width:147px" %)0%|(% style="width:146px" %)42.86%|(% style="width:185px" %)57.14%|(% style="width:175px" %)208
116 +
117 +|(% style="width:330px" %)**Tasks**|(% style="width:523px" %)**Parts where people struggled**
118 +|(% style="width:330px" %)Add a reminder|(% style="width:523px" %)(((
119 +* Don't know where to start
120 +* No immediate audio feedback indicating success
121 +)))
122 +|(% style="width:330px" %)Create a personal profile|(% style="width:523px" %)(((
123 +* It's hard to type with taped fingers
124 +)))
125 +|(% style="width:330px" %)Play memory game|(% style="width:523px" %)(((
126 +* Have no idea how to play the game, sometimes even after listening to the explicit instructions
127 +* There are too many words in the text
128 +* The beta version has no right or wrong prompts, different from the instructions, making people confused
129 +)))
130 +
105 105  = 4. Discussion =
106 106  
133 +=== Limitations: ===
107 107  
135 +We ran into some problems while creating the application and performing the experiment:
136 +
137 +==== Implementation: ====
138 +
139 +* We could not adapt the robot to the PwD due to time constraints. This means that we did not take into account the severity of the PwD's visual, acoustic and kinesthetic limitations while setting up Pepper.
140 +* We could not test the full capabilities of the robot due to privacy constraints. Since we fabricated the information about relatives to protect the privacy of participants, we were not able to perform the scenarios in a realistic manner.
141 +* Since the version of the Google Chrome browser on the Pepper tablet was outdated, we were not able to load our Flutter application onto it and simulate actual scenarios.
142 +* Participants were from a wide variety of different backgrounds and mother tongues, it was therefore not possible to adjust Pepper to the specific culture of the participant.
143 +
144 +==== Evaluation: ====
145 +
146 +* It is important to note that the participants who took part in our experiment were not affected by dementia.
147 +* The experiment was not conducted in a real-world setting.
148 +* We were not able to test the improvement in memory recall since it involves sustained interactions with a particular user over time and our experiment involved brief, one-time interactions.
149 +
150 +=== Future Improvements: ===
151 +
152 +* We can make our system more realistic/adapted to PwD by incorporating human-like responses, gestures and movements to Pepper.
153 +* We can make our system fully gesture/voice controlled to enable the PwD to (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)use the system without assistance from a caregiver, increasing their autonomy.
154 +* (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)We can incorporate privacy protocols like voice authentication and gaze detection to ensure that all personal information about the PwD, relatives and caregivers are kept safe and confidential.
155 +
108 108  = 5. Conclusions =
157 +
158 +After performing the experiment and running various statistical tests on the results obtained, we have made the following conclusions that hopefully answer some of our research questions:
159 +
160 +1. We believe that an information support application **DOES IMPROVE** a PwD's well-being, since it can (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)provide them with access to important information and support, improving their overall quality of life.
161 +1. We believe that a robot assistant **DOES IMPROVE** the experience of a PwD using it. The robot(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %) can provide companionship and assistance, making them feel independent and less isolated.
162 +
163 +(% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)While our experiment had its limitations, we believe that it provides a foundation for future research in developing personalized memory robots for people with dementia. We also believe that our research is applicable to mobile agents which increases the accessibility of the solution.
164 +
165 +
chart.png
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.KarthikPrakash
Size
... ... @@ -1,0 +1,1 @@
1 +15.4 KB
Content
XWiki.XWikiComments[5]
Author
... ... @@ -1,1 +1,0 @@
1 -Anonymous
Comment
... ... @@ -1,8 +1,0 @@
1 -interview with the person who interact with the tablet only:
2 -
3 -* The system takes time to learn how to use
4 -* prefer audio instructions
5 -* expect audio feedback after completing each task
6 -* prefer visualization over text
7 -
8 -Suggestion: it's better to weaken the existence of the system and make it easier to learn
Date
... ... @@ -1,1 +1,0 @@
1 -2023-04-01 10:17:27.32
XWiki.XWikiComments[7]
Author
... ... @@ -1,1 +1,0 @@
1 -Anonymous
Comment
... ... @@ -1,4 +1,0 @@
1 -H0: The robot can increase user satisfaction.
2 -
3 -H0: The robot can increase the usability of the system.
4 -
Date
... ... @@ -1,1 +1,0 @@
1 -2023-04-07 19:17:27.166