Skip to Content

Wiki source code of 4. Evaluation Methods

Last modified by Deepali Prabhu on 2023/04/10 15:02
Hide last authors
Deepali Prabhu 2.1 1 We follow the DECIDE[7] framework as a foundation to develop our evaluation specification.
2
3
4 We start by identifying the primary goals of our social robot:
5
6 * Short term :
7 ** Assist the PwD with the task of painting and ensure engagement in the task.
8 ** Reduce agitation and improve overall mood
9 * Long term : Improve quality of life.
10
11 Due to restrictions in time, we would need to conduct an in between study and collect pre, in between and post session metrics to determine the usability of our social robot. Additionally, since the participants would consist of fellow researchers and not actual dementia patients, we will have to rule out most commonly used scales like QUALID,QoL[1], BPSD, CMAIS [2]. Before even measuring the effectiveness of our social robot, it would it useful to collect metrics to gauge general attitudes towards robots of the participant before and after the session to gauge the technology acceptance. The General Attitudes Towards Robots Scale (GAToRS)[3] can be used for this.
12
13 Engagement level with the social robot can be measured by metrics like Free choice period and the User Enhanced User Engagement Scale[4]. The mood enhancement of users is also a key factor to assess. The Visual Analog Mood Scales (VAMS) would be an appropriate mood assessment scale for the elderly, due to its ease of understanding and use and its aptness for neurologically impaired patients[5].
14
15 There are several HRI frameworks to determine the general likability, usability and effectiveness of our Social Robot like Godspeed, SUS.
16
Deepali Prabhu 8.1 17 However we decide on using the **Physical Activity Enjoyment scale(PACES)[8]** with our proxy participants as it is most suitable for our use case and socio cognitive theories included in our design.
Deepali Prabhu 2.1 18
Deepali Prabhu 3.1 19
20
21
Deepali Prabhu 4.1 22
Deepali Prabhu 3.1 23 References
24
Deepali Prabhu 2.1 25 [1] Ready RE, Ott BR. Quality of Life measures for dementia. Health Qual Life Outcomes. 2003 Apr 23;1:11. doi: 10.1186/1477-7525-1-11. PMID: 12740036; PMCID: PMC155631.
26
27 [2] Yu C, Sommerlad A, Sakure L, Livingston G. Socially assistive robots for people with dementia: Systematic review and meta-analysis of feasibility, acceptability and the effect on cognition, neuropsychiatric symptoms and quality of life. Ageing Res Rev. 2022 Jun;78:101633. doi: 10.1016/j.arr.2022.101633. Epub 2022 Apr 21. PMID: 35462001.
28
29 [3]Koverola, M., Kunnari, A., Sundvall, J. //et al.// General Attitudes Towards Robots Scale (GAToRS): A New Instrument for Social Surveys. //Int J of Soc Robotics// **14**, 1559–1581 (2022). [[https:~~/~~/doi.org/10.1007/s12369-022-00880-3>>https://doi.org/10.1007/s12369-022-00880-3]]
30
31 [4] Heather L. O’Brien, Paul Cairns, Mark Hall,
32 A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form,
33 International Journal of Human-Computer Studies,
34
35 [5] Saeed Jaydari Fard, Andrew P. Lavender. (2019) [[A comparison of task-based mental fatigue between healthy males and females>>url:https://www.tandfonline.com/doi/abs/10.1080/21641846.2019.1562582]]. //Fatigue: Biomedicine, Health & Behavior// 7:1, pages 1-11.
36
37 [7] Kurniawan, S. (2004). Interaction design: Beyond human-computer interaction by Preece, Sharp, and Rogers (2001), ISBN 0471492787.
Deepali Prabhu 5.1 38
39 [8] Kendzierski, Deborah & DeCarlo, Kenneth. (1991). Physical Activity Enjoyment Scale: Two Validation Studies. Journal of Sport & Exercise Psychology. 13. 50-64. 10.1123/jsep.13.1.50.