Changes for page Measuring Instruments
Last modified by Mathieu Jung-Muller on 2022/04/04 13:37
From version 
6.1 
6.1
edited by Haoran Wang
on 2022/03/17 08:56
on 2022/03/17 08:56
Change comment:
There is no comment for this version
To version 9.1
9.1
edited by Mathieu Jung-Muller
on 2022/04/04 13:37
on 2022/04/04 13:37
Change comment:
There is no comment for this version
Summary
Details
-
Page properties
-
- Author
-
... ... @@ -1,1 +1,1 @@ 1 -XWiki. haoranwang1 +XWiki.Mathieu - Content
-
... ... @@ -2,42 +2,84 @@ 2 2 3 3 Evaluation is an important part of product design and it can last from the beginning to the very end. In the Human-Computer Interaction field, product evaluation can help researchers to identify good and bad designs, determine how usable features are, discover new features that were neglected before, and compare design choices to assist us in making decisions. 4 4 5 -= Frameworks= 6 -== DECIDE Framework== 7 -**D**etermine the goals 5 += Frameworks = 8 8 7 +== DECIDE Framework == 9 9 10 - ==== Explore thequestions====9 +**D**etermine the goals 11 11 11 +* What are the high-level goals of the evaluation? 12 +* Who wants it and why? 13 +* The goals influence the approach used for the study. 14 +In our evaluation, our goals are to check if the different stakeholders are able to use our prototype smoothly. Investigate how Pepper affects stakeholders' lives and try to use evaluation to improve our prototype. 12 12 13 -==== Choose the evaluation approach and methods ==== 16 +**E**xplore the questions 17 + Define goals and research questions. Our research questions are: 14 14 19 +* Are the different stakeholders able to use our prototype smoothly? 20 +* Does the prototype allow the PwD greater autonomy in their day-to-day life? 21 +* Does the prototype improve the emotional state of the PwD and their relatives? 15 15 16 -==== Identify the practical issues ==== 23 +**C**hoose the evaluation approach and methods 24 + The evaluation approach influences the methods used, and in turn, how data is collected, analyzed, and presented. 17 17 18 18 19 -==== Decide how to deal with ethical issues ==== 27 +**I**dentify the practical issues 28 + In our case, the most important practical issue is to gather our classmates to do the evaluation. We do not have any real dented people to evaluate. Besides, we have to make a schedule about when to evaluate our prototype. 20 20 30 +**D**ecide how to deal with ethical issues 31 + Ethical issues are the basis of the evaluation. We would inform all participants about practical issues and make sure to get their consent before starting the evaluation. Users have the right to know their tasks, know what will happen to the collected data, stop participation and leave when they wish. 21 21 22 -==== Evaluate, analyze, interpret and present the data ==== 33 +**E**valuate, analyze, interpret and present the data 34 + How data is evaluated, analyzed, interpreted, and presented. To make the results reliable and valid, we have to consider biases, reliability, validity, scope, and ecological validity. 23 23 36 +== IMPACT Framework == 37 +**I**ntention: Clarify objectives and hypotheses/claims 38 +**M**etrics & Measures: What, how and why 39 +**P**eople: Target group & participants 40 +**A**ctivities: Derive activities from use cases 41 +**C**ontext: Social, ethical, physical, etc. aspects 42 +**T**echnologies: Hardware and software 24 24 25 -== IMPACT Framework== 26 -==== Intention ==== 27 27 45 +== Formative Evaluation == 46 +Focus on the various processes of the human-technology interaction 47 +Derive open questions from the design specification. 28 28 29 -==== Metrics & Measures ==== 49 +== Summative Evaluation == 50 +Focuses on the overall effects of the human-technology interaction 51 +Specify research questions and hypotheses based on claims. 30 30 31 31 32 -= ===People====54 += Data = 33 33 56 +== Qualitative Data == 57 +Explore, discover, instruct 58 +* Understand and interpret interactions 59 +* Gain insight into views and perspectives 60 +* Open-ended, like interviews and participant observations 61 +* Try to identify patterns, features, themes 62 +* Study groups tend to be smaller 34 34 35 -==== Activities ==== 64 +== Quantitative Data == 65 +Describe, explain, predict 66 +* Measure outcomes, test hypotheses, and make predictions 67 +* Precise measurements 68 +* Identify statistical relationships 69 +* Larger number of participants 36 36 71 += Experimental Design = 72 +== Within-subjects == 73 +Each participant, all conditions 74 +* Few subjects needed 75 +* Reduced variability 76 +* More statistical power 77 +* Practice/fatigue effects 37 37 38 -==== Context ==== 39 - 40 - 41 -==== Technologies ==== 42 - 43 - 79 +== Between-subjects == 80 +Each participant, one condition 81 +* Simplicity 82 +* Less chance of practice/fatigue effects 83 +* More time, effort and participants 84 +* Individual variability 85 +* Environment factors