Skip to Content

Wiki source code of 3. Evaluation

Version 7.1 by William OGrady on 2024/03/25 14:05
Hide last authors
William OGrady 7.1 1 === In //Evaluation, //~-~- ===
2
3
William OGrady 6.1 4 (% class="wikigeneratedid" %)
5 [[a. Prototype>>https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2024group01/view/3.%20Evaluation/a.%20Prototype/]]
6
7 (% class="wikigeneratedid" %)
8 [[b. Test>>https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2024group01/view/3.%20Evaluation/b.%20Test/]]
9
10 (% class="wikigeneratedid" %)
11 [[c. Pilot Study>>https://xwiki.ewi.tudelft.nl/xwiki/wiki/sce2024group01/view/3.%20Evaluation/Pilot%20study/]]
12
13 === ===
14
Pravesha Ramsundersingh 5.1 15 === Setup ===
16
17 **Concrete Assignment: **The experiment will consist of groups of 3-4 people that will be given the opportunity to interact with NAO in a time-slot of 10 mins. In this time, the participants will be tasked to retrieve certain information. Due to time and language constraints of the NAO, we are unable to test Dutch PwDs. To simulate the effect of early-stage dementia symptoms, the NAO consists of data with a pre-defined persona. The participants will be tasked to retrieve key elements of the persona by interacting with the NAO. Seeing as this is not their own history, it will be as if they have 'forgotten' this information. After the 10 minutes of interaction with NAO, the test group will be given two evaluation surveys. The first evaluation survey is to see how much of the information was retained and discovered of the persona through the use of the NAO. The second evaluation survey is to allow the participant to indicate the emotional experience with the NAO.
18
19
Kris van Melis 2.1 20 NOTES;
21
22
23 We are testing out use-case 2.
24
25
26 Setup:
27
28 We get random test persons. The person get the role of someone with early stage dementia who has forgotten relationship (we might have to specifically focus on memory).
29
30
31 the person's goal is to find out as much information as possible from the memory bank of a person don't know.
32
33
34 we mainly test if the robot and the implemtation is effective in memory retrieval.
William OGrady 3.1 35
36
Kris van Melis 4.1 37 We ONLY test with students, not PwD.
William OGrady 3.1 38
39
40 PILOT STUDY!!!
Kris van Melis 4.1 41
42
43
44
45 Task list:
46
47 0. What do we want to evaluate?
48
49 a. work out what use case we want to evaluate and what assumptions/claim we want to test.
50
51
52 ~1. Write down test proposal & content
53
54 a. introduction information
55
56 b. how it gets processed
57
58 c. informed consent and they can leave any time etc
59
60 d. questions for evaluation of system
61
62 e. data analysis we are going to do
63
64 2. Ethical approval from that one board at the TU Delft
65
66 3. implement test for robot
67
68 4. evaluate with students
69
70
71