Changes for page b. Test

Summary

• Page properties (1 modified, 0 added, 0 removed)

Details

Page properties

Content

...	...	@@ -9,7 +9,7 @@
9	9
10	10	=== 2. Method ===
11	11
12		-[[Figure: procedure of the controlled experiment>>image:Procedure.png]]
	12	+[[Figure 2: procedure of the controlled experiment>>image:Procedure.png]]
13	13
14	14	==== 2.1 Participants ====
15	15
...	...	@@ -18,13 +18,13 @@
18	18
19	19	==== 2.2 Materials ====
20	20
21		-Materials for the test group include the NAO robot, an OpenAI chatbot running on a mobile phone, and a laptop to control the human-like movements of NAO, as described in the [[prototype>>doc:3\. Evaluation.a\. Prototype.WebHome]]. The control group requires written out text prompts instead.
	21	+Materials for the test group include the NAO robot, an OpenAI chatbot running on a mobile phone, and a laptop to control the human-like movements of NAO, as described in the [[prototype>>doc:3\. Evaluation.a\. Prototype.WebHome]]. The control group requires written-out text prompts instead.
22	22	Both groups require a set of fabricated memories, memory quizzes, writing materials, and evaluation forms for feedback. ** **
23	23
24	24
25	25	==== 2.3 Measures ====
26	26
27		-Memory retrieval success will be measured using a [[quiz>>doc:.Quiz.WebHome]] based on the fabricated memories. Participant feedback and observational notes will also be collected to assess the user experience and interaction effectiveness. We will evaluate the recall and memory-efficacy of both groups and try to relate the given answers to values** **such as autonomy and relatedness.
	27	+Memory retrieval success will be measured using a [[quiz>>doc:.Quiz.WebHome]] based on the fabricated memories. Participant feedback and observational notes will also be collected to assess the user experience and interaction effectiveness. We will evaluate the recall and memory efficacy of both groups and try to relate the given answers to values** **such as autonomy and relatedness.
28	28
29	29
30	30	==== 2.4 Procedure ====
...	...	@@ -65,35 +65,47 @@
65	65
66	66	=== 3. Results ===
67	67
68		-Twenty University students participated in the controlled experiment (M= 45%, F=55%). After gaining consent through the consent form, all participants were briefed to act as if experiencing the interaction from the viewpoint of a PwD. From the sample, half experienced
	68	+Twenty University students participated in the controlled experiment (M= 45%, F=55%). After gaining consent through the consent form, all participants were briefed to act as if experiencing the interaction from the viewpoint of a PwD. From the sample, half were asked to interact with the NAO and half were only using a voice-based version of the encyclopedia prototype.
69	69
70		-[[Pie chart showing the gender distribution of our sample. / Boxplot chart showing the familiarity with robots and the attitude towards technology.>>image:gender & familiarity.png]]
	70	+The main finding of the experiment is on Relatedness as can be seen in Table 1: the increase of affect in both Arousal and Valence dimensions during the experiment with the test group was higher than the experiment with the control group. The control group mean values in arousal and valence show no significant change at all.
71	71
72	72
73		-Most resulting p values had an alpha above 0.05 and thus were not significant enough.
	73	+[[Figure 3: Pie chart showing the gender distribution of our sample. ~| Figure 4: Boxplot chart showing the familiarity with robots and the attitude towards technology.>>image:gender & familiarity.png]]
74	74
75		-[[Table showing the Affect variables>>image:Table of means.png||height="214" width="737"]]
76	76
	76	+
	77	+Competence. No significant effect on using the NAO (p > 0.05)
	78	+
	79	+Covariates. had no significant influence on recall score (p > 0.05)
	80	+
	81	+
	82	+For other analyses, most resulting p values had an alpha above 0.05 and thus were not significant enough.
	83	+
	84	+[[Table 1: showing the Affect variables>>image:Table of means.png||height="214" width="737"]]
	85	+
77	77	=== 4. Discussion ===
78	78
	88	+The NAO may positively affects participant’ s affective state. Yet the other results were not significant enough to make a valid decision about. There are a few possible reasons why that is that will be elaborated upon in this section.
	89	+
	90	+
79	79	**Sample Group and Size impact validity of the study**
80	80
81		-The ecological validity of the study is impacted by the fact that there were no PwD in our sample.The scope of the experiment was limited to TU Delft University students. That means that future research may benefit from a closer approach to an experiment which is closer to the experience of PwD. Moreover, the controlled experiment was restricted to a cohort of 20 participants, underscoring the potential for enhancing result validity through the utilization of a larger sample size
	93	+The ecological validity of the study is impacted by the fact that there were no PwD in our sample. The scope of the experiment was limited to TU Delft University students. That means that future research may benefit from a closer approach to an experiment which is closer to the experience of PwD. Moreover, the controlled experiment was restricted to a cohort of 20 participants, underscoring the potential for enhancing result validity through the utilization of a larger sample size
82	82
83	83
84	84	**//Participants don't know about points system so they didn't answer with "getting the most points" in mind//**
85	85
86		-//Ambiguities in the evaluation briefing has led to several aspects in the results that might misrepresent the participants' gathered knowledge. Points were awarded to the participant for certain key descriptors, each family member's role, occupation, likes, dislikes and so on. Of course, this wasn't known to the participant, so they might have omitted descriptors they deemed less important or trivial and therefore scored worse, even though they'd heard and remembered them.//
	98	+//Ambiguities in the evaluation briefing has led to several aspects in the results that might misrepresent the participants' gathered knowledge. Points were awarded to the participant for certain key descriptors, each family member's role, occupation, likes, dislikes and so on. Of course, this wasn't known to the participants, so they might have omitted descriptors they deemed less important or trivial and therefore scored worse, even though they'd heard and remembered them.//
87	87
88	88
89	89	//**Participants don't know which people or facts are important, so they can get stuck in spots that are unrewarded**//
90	90
91		-//The choice to create a sprawling, multi-faceted database also had the side-effect of participants finding out a lot of information that was not rewarded by the grading system in any way. For example, the user can ask the robot to elaborate about certain memories or character traits of family members. There are also people in the database that act as ancillary characters and to create a sense of realism to the database, but participants can likely get stuck on learning about them as there is no implied hiearchy of importance to the participant. //
	103	+//The choice to create a sprawling, multi-faceted database also had the side-effect of participants finding out a lot of information that was not rewarded by the grading system in any way. For example, the user can ask the robot to elaborate on certain memories or character traits of family members. There are also people in the database that act as ancillary characters and to create a sense of realism to the database, but participants can likely get stuck on learning about them as there is no implied hierarchy of importance to the participant. //
92	92
93	93
94		-**//GPT Assistant can elaborate on any question, and therefore the user does not know what belongs to the database, and don't know where to focus//**
	106	+**//GPT Assistant can elaborate on any question, and therefore the user does not know what belongs to the database and don't know where to focus//**
95	95
96		-//Another limiation of the evaluation is with the GPT Assistant's ability to consistently elaborate on any question posed by the participant.//
	108	+//Another limitation of the evaluation is with the GPT Assistant's ability to consistently elaborate on any question posed by the participant.//
97	97
98	98
99	99
...	...	@@ -102,5 +102,3 @@
102	102	===== //**Ethical Considerations**// =====
103	103
104	104	We commit to high ethical standards, respecting the sensitive nature of simulating dementia conditions, and ensuring the well-being and dignity of all participants throughout the study. To ensure this we present a form containing the ethical considerations to each participant.
105		-
106		-