| ... |
... |
@@ -20,7 +20,7 @@ |
| 20 |
20 |
|
| 21 |
21 |
== 2.1 Participants == |
| 22 |
22 |
|
| 23 |
|
-All students in CS4235 Socio-Cognitive Engineering (2022-2023) in TU Delft are invited to test the robot. In the end, X students are presented. |
|
23 |
+All students in CS4235 Socio-Cognitive Engineering (2022-2023) in TU Delft are invited to test the robot. In the end, 20 students are presented. |
| 24 |
24 |
|
| 25 |
25 |
== 2.2 Experimental design == |
| 26 |
26 |
|
| ... |
... |
@@ -87,8 +87,25 @@ |
| 87 |
87 |
|
| 88 |
88 |
== 2.4 Measures == |
| 89 |
89 |
|
| 90 |
|
-Quantitative measures are used in a user evaluation. If a respondent had a minimum total score of 60% or more, he or she was considered to be satisfied with the application. |
|
90 |
+Two quantitative measures are used in the user evaluation: the first is to test different attributes including accessibility, trustworthiness, and comprehensibility; and the second is System Usability Scale (SUS). |
| 91 |
91 |
|
|
92 |
+**Interpretation for user evaluation ** |
|
93 |
+ |
|
94 |
+If a respondent had a minimum total score of 60% (15 out of 25 for the matrix question) or more, he or she was considered to be satisfied with the application. |
|
95 |
+ |
|
96 |
+**Scoring SUS** |
|
97 |
+ |
|
98 |
+* For odd items: subtract one from the user response. |
|
99 |
+* For even-numbered items: subtract the user responses from 5 |
|
100 |
+* This scales all values from 0 to 4 (with four being the most positive response). |
|
101 |
+* Add up the converted responses for each user and multiply that total by 2.5. This converts the range of possible values from 0 to 100 instead of from 0 to 40. |
|
102 |
+ |
|
103 |
+**Interpreting Scores for SUS [[*>>https://measuringu.com/sus/]]** |
|
104 |
+ |
|
105 |
+Interpreting scoring can be complex. The participant’s scores for each question are converted to a new number, added together and then multiplied by 2.5 to convert the original scores of 0-40 to 0-100. Though the scores are 0-100, these are not percentages and should be considered only in terms of their percentile ranking. |
|
106 |
+ |
|
107 |
+Based on research, a SUS score above a 68 would be considered above average and anything below 68 is below average, however, the best way to interpret your results involves “normalizing” the scores to produce a percentile ranking. |
|
108 |
+ |
| 92 |
92 |
== 2.5 Procedure == |
| 93 |
93 |
|
| 94 |
94 |
The procedure was conducted as follows: |
| ... |
... |
@@ -113,7 +113,15 @@ |
| 113 |
113 |
|
| 114 |
114 |
= 3. Results = |
| 115 |
115 |
|
|
133 |
+|=(% style="width: 199px;" %)Tasks|=(% style="width: 147px;" %)Succeded by Themselves|=(% style="width: 146px;" %)Succeded with Some Guidance|=(% style="width: 185px;" %)Succeded with Detailed Explicit Instructions|=(% style="width: 175px;" %)Average Time to Complete Task (s) |
|
134 |
+|(% style="width:199px" %)Turning the robot on and off|(% style="width:147px" %) |(% style="width:146px" %) |(% style="width:185px" %) |(% style="width:175px" %) |
|
135 |
+|(% style="width:199px" %)Add a reminder using "+" button|(% style="width:147px" %) |(% style="width:146px" %) |(% style="width:185px" %) |(% style="width:175px" %) |
|
136 |
+|(% style="width:199px" %)Add a reminder using voice|(% style="width:147px" %) |(% style="width:146px" %) |(% style="width:185px" %) |(% style="width:175px" %) |
|
137 |
+|(% style="width:199px" %)Check the reminder|(% style="width:147px" %) |(% style="width:146px" %) |(% style="width:185px" %) |(% style="width:175px" %) |
|
138 |
+|(% style="width:199px" %)Create a personal profile|(% style="width:147px" %) |(% style="width:146px" %) |(% style="width:185px" %) |(% style="width:175px" %) |
| 116 |
116 |
|
|
140 |
+if possible, note down some Parts Where Users Struggled in each task. |
|
141 |
+ |
| 117 |
117 |
= 4. Discussion = |
| 118 |
118 |
|
| 119 |
119 |
|
