Skip to Content

Wiki source code of 3. Evaluation

Version 5.1 by Pravesha Ramsundersingh on 2024/03/25 09:35
Show last authors
1 === Setup ===
2
3 **Concrete Assignment: **The experiment will consist of groups of 3-4 people that will be given the opportunity to interact with NAO in a time-slot of 10 mins. In this time, the participants will be tasked to retrieve certain information. Due to time and language constraints of the NAO, we are unable to test Dutch PwDs. To simulate the effect of early-stage dementia symptoms, the NAO consists of data with a pre-defined persona. The participants will be tasked to retrieve key elements of the persona by interacting with the NAO. Seeing as this is not their own history, it will be as if they have 'forgotten' this information. After the 10 minutes of interaction with NAO, the test group will be given two evaluation surveys. The first evaluation survey is to see how much of the information was retained and discovered of the persona through the use of the NAO. The second evaluation survey is to allow the participant to indicate the emotional experience with the NAO.
4
5
6 NOTES;
7
8
9 We are testing out use-case 2.
10
11
12 Setup:
13
14 We get random test persons. The person get the role of someone with early stage dementia who has forgotten relationship (we might have to specifically focus on memory).
15
16
17 the person's goal is to find out as much information as possible from the memory bank of a person don't know.
18
19
20 we mainly test if the robot and the implemtation is effective in memory retrieval.
21
22
23 We ONLY test with students, not PwD.
24
25
26 PILOT STUDY!!!
27
28
29
30
31 Task list:
32
33 0. What do we want to evaluate?
34
35 a. work out what use case we want to evaluate and what assumptions/claim we want to test.
36
37
38 ~1. Write down test proposal & content
39
40 a. introduction information
41
42 b. how it gets processed
43
44 c. informed consent and they can leave any time etc
45
46 d. questions for evaluation of system
47
48 e. data analysis we are going to do
49
50 2. Ethical approval from that one board at the TU Delft
51
52 3. implement test for robot
53
54 4. evaluate with students
55
56
57