| ... |
... |
@@ -95,8 +95,17 @@ |
| 95 |
95 |
|
| 96 |
96 |
=== (% style="color:inherit; font-family:inherit" %)Results of the survey:(%%) === |
| 97 |
97 |
|
| 98 |
|
-[[image:attach:chart.png||alt="Survey Results"]] |
|
98 |
+[[Figure: //Percentage of user satisfaction and SUS score//>>image:attach:chart.png]] |
| 99 |
99 |
|
|
100 |
+(% class="wikigeneratedid" %) |
|
101 |
+As mentioned earlier, the user evaluation incorporated two quantitative measures. The first measure evaluated the various attributes of the system, including accessibility, trustworthiness, perceivability, understandability, and empowerment. The second measure employed was the System Usability Scale (SUS). |
|
102 |
+ |
|
103 |
+(% class="wikigeneratedid" %) |
|
104 |
+The attributes-related evaluation was analyzed based on the following way: if a respondent had a minimum total score of 60% (15 out of 25) or more, he or she was considered to be satisfied with the application. 11 out of 14 (78.57%) of the users achieved a score of 15 or higher. The average score is 18. According to the standard operating protocol (Quintana et al., 2020), the feasibility test was to be considered successfully completed if at least 75% were satisfied with the use of the application. Therefore, based on this criterion, the feasibility test was considered successfully completed. |
|
105 |
+ |
|
106 |
+The System Usability Scale (SUS) was interpreted in terms of percentile ranking. The average SUS score for the stand-alone application is 54.17 (grade D), and that of the robot is 71.86 (grade B). Based on research, a SUS score above 68 would be considered above average and anything below 68 is below average. As a result, the usability of the robot was from this point of view considered successfully completed. |
|
107 |
+ |
|
108 |
+ |
| 100 |
100 |
|(% style="width:215px" %)**Attributes**|(% style="width:211px" %)**Mean (control group)**|(% style="width:229px" %)**Mean (Experimental group)**|(% style="width:197px" %)**P-value** |
| 101 |
101 |
|(% style="width:215px" %)Accessibility|(% style="width:211px" %)2,33|(% style="width:229px" %)3,25 |(% style="width:197px" %)0,0644 |
| 102 |
102 |
|(% style="width:215px" %)Trustability|(% style="width:211px" %)3,83|(% style="width:229px" %)4,125|(% style="width:197px" %)0,3165 |
| ... |
... |
@@ -105,6 +105,9 @@ |
| 105 |
105 |
|(% style="width:215px" %)Empowerment|(% style="width:211px" %)3,33|(% style="width:229px" %)4|(% style="width:197px" %)0,0895 |
| 106 |
106 |
|(% style="width:215px" %)Usability|(% style="width:211px" %)54,16666667|(% style="width:229px" %)71,875|(% style="width:197px" %)0,0903 |
| 107 |
107 |
|
|
117 |
+(% class="wikigeneratedid" %) |
|
118 |
+//Table: User evaluation score// |
|
119 |
+ |
| 108 |
108 |
=== Observation: (Total percentage sums up to 100) === |
| 109 |
109 |
|
| 110 |
110 |
|=(% style="width: 199px;" %)Tasks|=(% style="width: 147px;" %)Succeeded by Themselves|=(% style="width: 146px;" %)Succeeded with Some Guidance|=(% style="width: 185px;" %)Succeeded with Detailed Explicit Instructions|=(% style="width: 175px;" %)Average Time to Complete Task (s) |
| ... |
... |
@@ -114,6 +114,9 @@ |
| 114 |
114 |
|(% style="width:199px" %)Verify current profiles|(% style="width:147px" %)85.71%|(% style="width:146px" %)14.29%|(% style="width:185px" %)NA|(% style="width:175px" %)32 |
| 115 |
115 |
|(% style="width:199px" %)Play memory game|(% style="width:147px" %)0%|(% style="width:146px" %)42.86%|(% style="width:185px" %)57.14%|(% style="width:175px" %)208 |
| 116 |
116 |
|
|
129 |
+//Table: Results of user performance of tasks // |
|
130 |
+ |
|
131 |
+ |
| 117 |
117 |
|(% style="width:330px" %)**Tasks**|(% style="width:523px" %)**Parts where people struggled** |
| 118 |
118 |
|(% style="width:330px" %)Add a reminder|(% style="width:523px" %)((( |
| 119 |
119 |
* Don't know where to start |
| ... |
... |
@@ -128,27 +128,22 @@ |
| 128 |
128 |
* The beta version has no right or wrong prompts, different from the instructions, making people confused |
| 129 |
129 |
))) |
| 130 |
130 |
|
|
146 |
+//Table: Difficulties that users struggled with when solving tasks// |
|
147 |
+ |
| 131 |
131 |
= 4. Discussion = |
| 132 |
132 |
|
| 133 |
|
-=== Limitations: === |
|
150 |
+(% class="wikigeneratedid" %) |
|
151 |
+In light of our research question, we found no notable disparities between the stand-alone application and the robot. The results may be influenced by the experimental setup. While we aimed to emulate a real-world scenario for participants to perform the tasks, their pre-existing digital device proficiency could have played a role. The participants may have had prior experience with using similar applications or robots, which could have affected their performance and perception of the two groups. Nevertheless, given that people with dementia likely have limited knowledge of utilizing mobile devices, we maintain a positive outlook on the potential efficacy of the visual-audio aid provided by the robot to enhance the experience of PwDs utilizing the application. |
| 134 |
134 |
|
| 135 |
|
-We ran into some problems while creating the application and performing the experiment: |
|
153 |
+==== Limitations: ==== |
| 136 |
136 |
|
| 137 |
|
-==== Implementation: ==== |
| 138 |
|
- |
| 139 |
139 |
* We could not adapt the robot to the PwD due to time constraints. This means that we did not take into account the severity of the PwD's visual, acoustic and kinesthetic limitations while setting up Pepper. |
| 140 |
140 |
* We could not test the full capabilities of the robot due to privacy constraints. Since we fabricated the information about relatives to protect the privacy of participants, we were not able to perform the scenarios in a realistic manner. |
| 141 |
141 |
* Since the version of the Google Chrome browser on the Pepper tablet was outdated, we were not able to load our Flutter application onto it and simulate actual scenarios. |
| 142 |
142 |
* Participants were from a wide variety of different backgrounds and mother tongues, it was therefore not possible to adjust Pepper to the specific culture of the participant. |
| 143 |
143 |
|
| 144 |
|
-==== Evaluation: ==== |
|
160 |
+==== Future Improvements: ==== |
| 145 |
145 |
|
| 146 |
|
-* It is important to note that the participants who took part in our experiment were not affected by dementia. |
| 147 |
|
-* The experiment was not conducted in a real-world setting. |
| 148 |
|
-* We were not able to test the improvement in memory recall since it involves sustained interactions with a particular user over time and our experiment involved brief, one-time interactions. |
| 149 |
|
- |
| 150 |
|
-=== Future Improvements: === |
| 151 |
|
- |
| 152 |
152 |
* We can make our system more realistic/adapted to PwD by incorporating human-like responses, gestures and movements to Pepper. |
| 153 |
153 |
* We can make our system fully gesture/voice controlled to enable the PwD to (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)use the system without assistance from a caregiver, increasing their autonomy. |
| 154 |
154 |
* (% style="color:#000000; font-family:Arial; font-size:11pt; font-style:normal; font-variant:normal; font-weight:400; text-decoration:none; white-space:pre-wrap" %)We can incorporate privacy protocols like voice authentication and gaze detection to ensure that all personal information about the PwD, relatives and caregivers are kept safe and confidential. |
