Skip to Content

Changes for page Measuring Instruments

Last modified by Sofia Kostakonti on 2022/04/04 12:04
From version 1.1 Icon
edited by Bart Vastenhouw
on 2022/02/07 21:34
Change comment: Imported from XAR
To version Icon 4.1 Icon
edited by Veikko Saikkonen
on 2022/03/01 11:59
Change comment: There is no comment for this version

Summary

Details

Icon Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -xwiki:XWiki.BartVastenhouw
1 +XWiki.VSaikkonen
Content
... ... @@ -1,0 +1,90 @@
1 +
2 += Frameworks =
3 +
4 +== DECIDE ==
5 +
6 +**D**etermine the goals
7 +- What are the high-level goals? Who wants it and why? The goals influence the approach of the study
8 +
9 +**E**xplore the questions
10 +- Define the goals and reseach questions
11 +
12 +**C**hoose evaluation approach and methods
13 +- Influences the data collection, analysis and presentation
14 +- Exploratory research => qualitative data, observational study,
15 +
16 +**I**dentify practical issues
17 +- Users, budget, schedule, equipment etc.
18 +- Pilot study important
19 +
20 +**D**ecide about ethical issues
21 +- Adhere to ethical procedure
22 +- User rights! Explain the goals (before/after), methodology etc.
23 +
24 +**E**valuate, analyze, interpret, present data
25 +- Reliable results: Can the results be replicated?
26 +- Validity: Is the data related to the hypothesis?
27 +- Bias: Are the results unbiased?
28 +- Scope: Can this be generalised?
29 +- Ecological validity: Is the environment influencing the results?
30 +
31 +== IMPACT ==
32 +
33 +**I**ntention: Present the objectives and claims
34 +
35 +**M**easures and metrics: "What, how and why"
36 +
37 +**P**eople: Define the participants
38 +
39 +**A**ctivities: Use cases into activities
40 +
41 +**C**ontext: Social, ethical, physical, etc. environment definition
42 +
43 +**T**echnologies: Hardware and software
44 +
45 +
46 += Evaluation methods =
47 +
48 +== Formative evaluation ==
49 +
50 +- Open ended evaluation on the design
51 +- E.g. How will the users respond to the new design?
52 +
53 +
54 +== Summative evaluation ==
55 +
56 +- Focus on the overall effect
57 +- Summarizes if the objective is reached
58 +- E.g. Are the participants happier when working with design X in comparison to design Y?
59 +
60 +== Data ==
61 +
62 +- Qualitative: Explore, discover, instruct
63 +
64 +- Quantitative: Describe, explain, predict
65 +
66 + - Subjective quantitative
67 +
68 +
69 +== Statistics ==
70 +
71 +- Descriptive: Describe the dataset, e.g. mean time on task
72 +- Inferential: Using a sample to infer about a population, e.g. predicted mean time on task based on user characteristics.
73 +
74 +== Experiment Design: Conditions ==
75 +
76 +=== Within Subjects (Repeated Measures) ===
77 +Each participant is subjected to all the test conditions.
78 +Fewer subjects needed and reduces variance in the Can be difficult to setup due to subjects fatiguing, learning about the setup or simply not having enough time.
79 +
80 +=== Between Subjects (Between Groups) ===
81 +One subject undergoes only one test. Simple to execute, but results in significant variance due to inter-subject differences in characteristics.
82 +
83 +
84 +== Lenses ==
85 +- Lense means to take different perspectives looking at your system
86 +- E.g. perspective of the stakeholders, other groups, or technical/legal
87 +
88 +
89 +
90 +