Changes for page Test

Summary

• Page properties (2 modified, 0 added, 0 removed)

Details

Page properties

Author

...	...	@@ -1,1 +1,1 @@
1		-XWiki.PietroPiccini
	1	+XWiki.Mathieu

Content

...	...	@@ -64,32 +64,32 @@
64	64	→ feelings from the PwD themselves
65	65	→ reports from relatives and HCP
66	66	These results will most likely never be yes-no results, but more like clues or hints that show whether some things worked on not, which will be the point of our discussion.
67		-NB: This part explains what we expect as a kind of result, it will be replaced by actual results after we perform an experiment with the class. There may also be interesting points we did not think about.
68	68
69	69	== Discussion ==
70	70
71	71	* Reliability: Yes. One could replicate the same experiment with other patients.
72		-* Validity: TBD.
73		-* Biases: TBD.
	71	+* Validity: TBD. (This this ideal evaluation has not been conducted, we cannot really estimate this.)
	72	+* Biases: TBD.(This this ideal evaluation has not been conducted, we cannot really estimate this.)
74	74	* Scope: No. It would be very difficult to generalize the results since each prototype is built for a special patient. However, if the results conclude that the customized prototypes did improve the well-being of the people, then similar efforts to customize Pepper for more patients should produce similar effects.
75	75	* Ecological validity: Yes. Since we compare "without Pepper" (BEFORE) and "with Pepper" (AFTER) in a similar environment (i.e., for everything but Pepper), the results are not dependent on the environment.
76	76
77	77	== Conclusions ==
78	78
	78	+(Since this ideal evaluation has not been conducted, we do not have conclusions to be drawn.)
79	79
80	80
	81	+
81	81	= Feasible evaluation (students) =
82	82
83	83	== Problem statement and research questions ==
84	84
85		-This project uses a Social Cognitive Engineering (SCE) approach to guide the design and research process. The SCE method provides a systematic approach to our study of robots for PwDs. The main goal of our application is to improve the well-being of the person with dementia (PwD) and of those living with them.
	86	+For the prototype that we have designed now, and for a smaller scope evaluation, the research questions are smaller-scope versions of our main research questions for the previous part.
	87	+We also cannot test all our claims during only a short evaluation session with students.
86	86
87		-For the prototype that we have designed now, these are some research questions that we want to address.
	89	+1. Are the participants able to use our prototype smoothly?
	90	+1. Does the prototype improve the autonomy of the patient on a specific activity?
	91	+1. Does the prototype improve the emotional state of the participant after completing a specific activity?
88	88
89		-1. Are the different stakeholders able to use our prototype smoothly?
90		-1. Does the prototype allow the PwD greater autonomy in their day-to-day life?
91		-1. Does the prototype improve the emotional state of the PwD and their relatives?
92		-
93	93	Based on these questions and our evaluation setting, we can evaluate some of our claims from our Claim page.
94	94
95	95	1. CL01: The PwD becomes aware of Pepper's presence.
...	...	@@ -107,22 +107,18 @@
107	107
108	108	== Method ==
109	109
110		-In our situation, we recruit 20 students in our class to simulate the research. Since they are not real PwD, we ask them to act as if they were in home settings and we observe their behavior and expression. These data will also be reviewed after the evaluation to obtain the data and feedback.
	110	+In order to collect qualitative data, we prepared two questionnaires to give to the participants: the affect assessment questionnaire and the system assessment questionnaire. All questionnaire questions are expressed in the form of statements and the participant can express one out of seven levels of agreement/disagreement from strongly disagree to strongly agree. The only exception is the additional remarks field at the end of the system assessment questionnaire, where the participants can freely put any comment/remark/feedback they have on the experiment.
111	111
112		-==== Questionnaires ====
113		-
114		-In order to collect qualitative data, we prepared two questionnaires to give to the participants: the affect assessment questionnaire and the system assessment questionnaire. All questionnaire questions are expressed in the form of statements and the participant can express one out of seven levels of agreement/disagreement from strongly disagree to strongly agree with the only exception being the additional remarks field at the end of the system assessment questionnaire.
115		-
116	116	**Affect assessment**
117	117
118		-The affect assessment questionnaire asks the participant to describe their mood and feelings experienced during a task. The questionnaire is given two times: once before the experiments and once right after. The purpose of this questionnaire is to compare the feelings experienced by the participant before the interaction with Pepper and after to study how Pepper influences the participant's feelings. The questionnaire has six questions each designed to measure the level of a certain feeling experienced at the moment of filling the questionnaire. The six feelings we take into consideration are evaluated from the following statements:
	114	+The affect assessment questionnaire asks the participant to describe their mood. The questionnaire is given two times: once before the experiments and once right after. The purpose of this questionnaire is to compare the mood experienced by the participant before the interaction with Pepper and after. The idea is to assess the effect of the activity. The questionnaire has six questions each designed to measure the level of a certain feeling experienced at the moment of filling the questionnaire. The six feelings we take into consideration are evaluated from the following statements:
119	119
120		-1. I feel sad
121		-1. I feel content
122		-1. I feel calm
123		-1. I feel tired
124		-1. I feel nervous
125		-1. I feel caring
	116	+1. I feel sad.
	117	+1. I feel content.
	118	+1. I feel calm.
	119	+1. I feel tired.
	120	+1. I feel nervous.
	121	+1. I feel caring.
126	126
127	127	**System assessment**
128	128
...	...	@@ -129,8 +129,8 @@
129	129	The system assessment questionnaire is given after the interaction with Pepper and its purpose is to assess the participant's experience with Pepper in more detail. The questionnaire has fourteen questions which are designed to answer research questions as described below:
130	130
131	131	1. I like gardening
132		-1. I think Pepper made the task easier for me.
133	133	1. I would have known how to do the whole task without Pepper.
	129	+1. I think Pepper made the task easier for me.
134	134	1. Pepper was easy to understand.
135	135	1. I enjoyed the task more than if I had had to do it alone.
136	136	1. I am pleased that Pepper reminded me to do the activity.
...	...	@@ -145,7 +145,7 @@
145	145
146	146	Research questions:
147	147
148		-1. To what extent did Pepper improve the task's experience? [Questions: 2,3,4,5,10,11,12]
	144	+1. To what extent did Pepper improve the task's experience? [Questions: 3,4,5,10,11,12]
149	149	1. To what extent did Pepper improve the participant's autonomy and perception of control? [Questions: 6, 8, 9]
150	150	1. To what extent did Pepper improve the participant's sense of accomplishment? [Questions: 7]
151	151	1. Did the participant perceive Pepper as a social agent? [Questions: 13, 14]
...	...	@@ -155,40 +155,38 @@
155	155
156	156	== Participants ==
157	157
158		-For this study, we simulate the real research by including 24 students from TU Delft. One third of them were female and two thirds male, so gender balance was decent. The students were not asked to roleplay someone with dementia, but simply to follow the experiment and do the activity.
	154	+In our situation, our participants are 24 students from TUD. Most of them come from Computer Science and/or from the SCE class. One third of them are female and two thirds male, so gender balance is decent.
	155	+Roleplaying PwDs may be difficult for students, especially for those who have not followed the course. For this reason, and in order not to introduce an extra bias "knows about dementia"/"does not know about dementia", we decided to ask the students to just act as students.
159	159
160	160	== Experimental design ==
161	161
162		-Since our evaluating process is relatively short, we use within-subject, which means each participant goes through all conditions. In this way, our experiment tends to have more statistical power and less variability. Furthermore, every PwD may have different issues in their day-to-day life, while also not having the same living conditions (alone, living with husband, family, etc), thus would require a different treatment. But in our setting, we use the same setting for every participant.
	159	+Since our evaluating process is relatively short, we use within-subject, which means each participant goes through all conditions. In this way, our experiment tends to have more statistical power and less variability.
163	163
164	164	== Tasks ==
165	165
166		-All participants will go through our designed testing process, which includes medication/meal/activity reminder and activity breakdown.
	163	+All participants will go through our designed testing process, which combines a calendar event reminder and an activity breakdown.
	164	+Pepper will propose the activity to the participant, then go step by step with them through the activity.
167	167
168		-Medication/meal/activity reminder: The robot will remind the patient of daily activities, through which we can see the effectiveness according to their reactions.
169		-Activity breakdown: In this part, the robot will break down some complex tasks into a list of simple tasks and PwDs can follow the steps to accomplish complex tasks.
170		-
171	171	== Measures ==
172	172
173		-We are planning to make behavioural and emotional measurements.
174		-Behavioral measurements are the actions that the PwD is going to perform during the week, so it can be considered as subjective quantitative data. This will involve the relatives, the HCP and the PwD themselves to quantify whether the use of Pepper did actually result in an increase in autonomy for the PwD.
175		-Emotional measurements are more related to state of mind, change of expression and mood, so it can be considered as qualitative data. This can be measured by frequent talks with the PwD, either by the relatives or the HCP.
176		-Measures will be done by oral discussions with the PwD, HCP and relatives.
	168	+We measure the mood of the participants through the affect assessment questionnaires. We also measure some aspects of the interaction between Pepper and the participant through the system assessment questionnaire. Both are qualitative data, as the participant gives an answer ranging from strongly disagree to strongly agree.
177	177
178	178	== Procedure ==
179	179
180		-At the start of the evaluation, all participants will be gathered in the same room, together with the researchers and relatives. We will explain the whole evaluation process, provide simple instructions, explain that participation in the evaluations is voluntary, and participants are free to stop the evaluation at any time. Then, we will explain how the gathered data will be analyzed and help us to improve our prototype. All data will be kept private. We will also emphasize that if there is anything the participants dislike, they should let us know. The purpose of the study is to find out what they think of the prototype, and their honesty is greatly appreciated. Then, we will ask them to sign the consent form.
181		-We will simulate a home setting, which is the most common scenario for PwDs. All PwDs are going to complete the evaluation separately.
	172	+Before the experiment, we explain to the participants the purpose of our evaluation (context of SCE class, our prototype).
	173	+We tell them that we will gather data in the questionnaires, that the data are fully anonymized and will only be used for our result analysis in this course.
	174	+We also briefly explain to them how Pepper works, especially that she only listens when the eyes are blue.
	175	+After the explanation, the participant can fill the consent form and the first affect questionnaire.
	176	+Then, the experiment starts. The experiment is conducted in autonomy between the participant and Pepper. We only intervene if asked by the participant or if something is going wrong with the experiment.
	177	+After the activity is finished by the participant, they fill the second affect questionnaire and the system assessment questionnaire.
	178	+After this is done, we thank them for their participation and the evaluation session is over.
182	182
183		-Our robot will give corresponding prompts. For example, “It’s time to have lunch”, “Medicine time!”, “Today is your birthday. Happy Birthday!”.
184		-We will record the reaction and behavior of participants after they heard the prompts.
185		-To test the activity breakdown function, our robot will choose a relatively complex task to perform, such as making a paper plane, doing some exercise. Our robot will break it down into simple steps.
186		-Record the reactions of participants and evaluate how is the activity accomplished.
187		-
188	188	== Material ==
189	189
190		-1. Consent form. To protect the privacy of participants and ensure the evaluation process goes smoothly, we will ask participants to sign a consent form, indicating they are willing to take part in the evaluation and the data gathered from the experiment will be analyzed by researchers.
	182	+1. Consent form. To protect the privacy of participants and ensure the evaluation process goes smoothly, we will ask participants to sign a consent form, indicating they are willing to take part in the evaluation and the data gathered from the experiment will be analyzed by researchers. The consent form is accessed online via a QR code.
	183	+1. Questionnaires. There are three questionnaires to answer (mood 1, mood 2, system). They are also accessed online via a QR code.
191	191	1. Pepper robot. Our robot is programmed using Choregraphe. The robot will have the same behaviour for every participant. However, the input data will be entered by the HCP (and potentially the relatives).
	185	+1. Gardening stuff. In order to conduct the activity, we bought and brought basic stuff required for gardening (soil, pot, seeds, etc).
192	192
193	193	== Results ==
194	194