Changes for page Measuring Instruments

Last modified by Sofia Kostakonti on 2022/04/04 12:04

From version Icon 2.1 Icon
edited by Veikko Saikkonen
on 2022/03/01 11:29
Change comment: There is no comment for this version
To version Icon 3.2 Icon
edited by Veikko Saikkonen
on 2022/03/01 11:56
Change comment: There is no comment for this version

Summary

Details

Icon Page properties
Content
... ... @@ -7,13 +7,78 @@
7 7  - What are the high-level goals? Who wants it and why? The goals influence the approach of the study
8 8  
9 9  **E**xplore the questions
10 +- Define the goals and reseach questions
10 10  
11 11  **C**hoose evaluation approach and methods
13 +- Influences the data collection, analysis and presentation
14 +- Exploratory research => qualitative data, observational study,
12 12  
13 13  **I**dentify practical issues
17 +- Users, budget, schedule, equipment etc.
18 +- Pilot study important
14 14  
15 15  **D**ecide about ethical issues
21 +- Adhere to ethical procedure
22 +- User rights! Explain the goals (before/after), methodology etc.
16 16  
17 17  **E**valuate, analyze, interpret, present data
25 +- Reliable results: Can the results be replicated?
26 +- Validity: Is the data related to the hypothesis?
27 +- Bias: Are the results unbiased?
28 +- Scope: Can this be generalised?
29 +- Ecological validity: Is the environment influencing the results?
18 18  
31 +== IMPACT ==
19 19  
33 +**I**ntention: Present the objectives and claims
34 +
35 +**M**easures and metrics: "What, how and why"
36 +
37 +**P**eople: Define the participants
38 +
39 +**A**ctivities: Use cases into activities
40 +
41 +**C**ontext: Social, ethical, physical, etc. environment definition
42 +
43 +**T**echnologies: Hardware and software
44 +
45 +
46 += Evaluation methods =
47 +
48 +== Formative evaluation ==
49 +
50 +- Open ended evaluation on the design
51 +- E.g. How will the users respond to the new design?
52 +
53 +
54 +== Summative evaluation ==
55 +
56 +- Focus on the overall effect
57 +- Summarizes if the objective is reached
58 +- E.g. Are the participants happier when working with design X in comparison to design Y?
59 +
60 +== Data ==
61 +
62 +- Qualitative: Explore, discover, instruct
63 +
64 +- Quantitative: Describe, explain, predict
65 +
66 + - Subjective quantitative
67 +
68 +
69 +== Statistics ==
70 +
71 +- Descriptive: Describe the dataset, e.g. mean time on task
72 +- Inferential: Using a sample to infer about a population, e.g. predicted mean time on task based on user characteristics.
73 +
74 +== Experiment Design: Conditions ==
75 +
76 +=== Within Subjects (Repeated Measures) ===
77 +Each participant is subjected to all the test conditions.
78 +Fewer subjects needed and reduces variance in the Can be difficult to setup due to subjects fatiguing, learning about the setup or simply not having enough time.
79 +
80 +=== Between Subjects (Between Groups) ===
81 +One subject undergoes only one test. Simple to execute, but results in significant variance due to inter-subject differences in characteristics.
82 +
83 +
84 +