3. Evaluation Methods

1

To ground the design rationale in practice, the prototype of the NAO for the PwD will be evaluated in a formative evaluation with the PwD and their caregivers. The evaluation will investigate the process of how interaction with the NAO can alleviate potential symptoms caused by early-stage dementia in PwD. Three methods of evaluation will be applied to have reliable and accurate results, as Bethel. et al (2020)[1] suggests.

2

3

The study will focus on the prototype's potential effects, which are based on the desired value of autonomy as a part of self-direction. Before the study, possible confounding variables need to be examined such as the PwD's attitude towards robots and the **autonomy, relatedness** and **competence**.

==== ====

== //**Study design variables**// ==

8

9

The study will hypothesize on the following variables with regards to the system:

10

11

12

1. **Autonomy. **Does the design increase the sense of //autonomy //in a PwD?

13

1. **Relatedness. **Does the design positively affect the PwD's //affective //state?

14

11. **Affect**. How do participants feel about using the robot in this state?

15

11. **Attitude towards Technology**. How do people think about using technology? Are they biased towards the robot before the study?

16

1. **Competence. **Is the design //competent//; is the design capable enough for the PwD to rely on it?

17

11. **Memory self-efficacy **(pre-study) How good are participants at remembering information?

18

11. **Memory recall **(post-study) Can the participant accurately retrieve information through the robot?

19

20

21

For a sample size as small as 20 participants, it is most adequate to apply a within-subjects design (they require fewer participants) [1]. That means there is an approach where every PwD is experiencing all of the conditions examined. A within-subjects design might be prone to confounds such as pre-existing notions in the environment. That is why the attitude towards robots and the pre-study sense of affect and autonomy should be examined and evaluated as such.

22

23

[[image:Socio-Cognitive Engineering - Frame 1(4).jpg]]

=== Surveys ===

For autonomy, a validated survey tool is the self-care score as used in the Diabetes research paper of PAL by Neerincx et al.

29

30

Affect can be measured with a button ([[http:~~/~~/ii.tudelft.nl/~~~~joostb/affectbutton_version2_original.html>>url:http://ii.tudelft.nl/~~joostb/affectbutton_version2_original.html]]) that takes less than a minute. It will be used for both pre-experiment and post-experiment evaluation.

31

32

For Attitude towards robots, we build upon the works of [reference] and create a set of two 1-minute questions.

33

34

A well-suited tool for Memory self-efficacy is the Rivermead behavioural memory test [reference]. We adopt it to fit to the ecological validity and domain of interest in our study.

35

36

37

== //**References**// ==

38

39

[1] Bethel, C.L., Henkel, Z., Baugus, K. (2020). Conducting Studies in Human-Robot Interaction. In: Jost, C., //et al.// Human-Robot Interaction. Springer Series on Bio- and Neurosystems, vol 12. Springer, Cham. https:~/~/doi.org/10.1007/978-3-030-42307-0_4

Wiki source code of 3. Evaluation Methods