Skip to Content

Changes for page b. Test

Last modified by Demi Tao on 2023/04/10 10:13
From version 3.1
edited by Arber Demi
on 2023/03/25 17:08
Change comment: There is no comment for this version
To version 7.1
edited by Demi Tao
on 2023/03/27 14:31
Change comment: There is no comment for this version

Summary

Details

Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.ArberDemi
1 +XWiki.DemiTao
Content
... ... @@ -20,7 +20,7 @@
20 20  
21 21  == 2.1 Participants ==
22 22  
23 -All students in CS4235 Socio-Cognitive Engineering (2022-2023) in TU Delft are invited to test the robot. In the end, X students are presented.
23 +All students in CS4235 Socio-Cognitive Engineering (2022-2023) in TU Delft are invited to test the robot. In the end, 20 students are presented.
24 24  
25 25  == 2.2 Experimental design ==
26 26  
... ... @@ -87,8 +87,25 @@
87 87  
88 88  == 2.4 Measures ==
89 89  
90 -Quantitative measures are used in a user evaluation.  If a respondent had a minimum total score of 60% or more, he or she was considered to be satisfied with the application.
90 +Two quantitative measures are used in the user evaluation: the first is to test different attributes including accessibility, trustworthiness, and comprehensibility; and the second is System Usability Scale (SUS).
91 91  
92 +**Interpretation for user evaluation **
93 +
94 +If a respondent had a minimum total score of 60% (15 out of 25 for the matrix question) or more, he or she was considered to be satisfied with the application.
95 +
96 +**Scoring SUS**
97 +
98 +* For odd items: subtract one from the user response.
99 +* For even-numbered items: subtract the user responses from 5
100 +* This scales all values from 0 to 4 (with four being the most positive response).
101 +* Add up the converted responses for each user and multiply that total by 2.5. This converts the range of possible values from 0 to 100 instead of from 0 to 40.
102 +
103 +**Interpreting Scores for SUS [[*>>https://measuringu.com/sus/]]**
104 +
105 +Interpreting scoring can be complex. The participant’s scores for each question are converted to a new number, added together and then multiplied by 2.5 to convert the original scores of 0-40 to 0-100.  Though the scores are 0-100, these are not percentages and should be considered only in terms of their percentile ranking.
106 +
107 +Based on research, a SUS score above a 68 would be considered above average and anything below 68 is below average, however, the best way to interpret your results involves “normalizing” the scores to produce a percentile ranking.
108 +
92 92  == 2.5 Procedure ==
93 93  
94 94  The procedure was conducted as follows:
... ... @@ -113,7 +113,15 @@
113 113  
114 114  = 3. Results =
115 115  
133 +|=(% style="width: 199px;" %)Tasks|=(% style="width: 147px;" %)Succeded by Themselves|=(% style="width: 146px;" %)Succeded with Some Guidance|=(% style="width: 185px;" %)Succeded with Detailed Explicit Instructions|=(% style="width: 175px;" %)Average Time to Complete Task (s)
134 +|(% style="width:199px" %)Turning the robot on and off|(% style="width:147px" %) |(% style="width:146px" %) |(% style="width:185px" %) |(% style="width:175px" %)
135 +|(% style="width:199px" %)Add a reminder using "+" button|(% style="width:147px" %) |(% style="width:146px" %) |(% style="width:185px" %) |(% style="width:175px" %)
136 +|(% style="width:199px" %)Add a reminder using voice|(% style="width:147px" %) |(% style="width:146px" %) |(% style="width:185px" %) |(% style="width:175px" %)
137 +|(% style="width:199px" %)Check the reminder|(% style="width:147px" %) |(% style="width:146px" %) |(% style="width:185px" %) |(% style="width:175px" %)
138 +|(% style="width:199px" %)Create a personal profile|(% style="width:147px" %) |(% style="width:146px" %) |(% style="width:185px" %) |(% style="width:175px" %)
116 116  
140 +if possible, note down some Parts Where Users Struggled in each task.
141 +
117 117  = 4. Discussion =
118 118  
119 119  
XWiki.XWikiComments[2]
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.MarkNeerincx
Comment
... ... @@ -1,0 +1,5 @@
1 +Refer explicitly to the claims in the specification section ("claims we are looking to test with this testing procedure are related to the functionality and usability" is far too general).
2 +
3 +It is a good start, but try to be more specific (based on this information, other researchers should be able to replicate the test. E.g., how many participants do you plan to include?
4 +
5 +Good that you think about how the participant can play-act as a PwD. Do not put it in footnotes, but make a concise paragraph of it (it is an important part of your test method).
Date
... ... @@ -1,0 +1,1 @@
1 +2023-03-26 16:49:48.530