Skip to Content

Changes for page Test

Last modified by Sofia Kostakonti on 2022/04/05 14:08
From version Icon 111.1 Icon
edited by Sofia Kostakonti
on 2022/04/04 19:31
Change comment: There is no comment for this version
To version Icon 118.1 Icon
edited by Sofia Kostakonti
on 2022/04/04 20:32
Change comment: There is no comment for this version

Summary

Details

Icon Page properties
Content
... ... @@ -27,11 +27,11 @@
27 27  
28 28  == Experimental design ==
29 29  
30 -For the experiment, we used a within-subject design. All of the participants interacted with both versions of the robot, with half of the participants interacting with version 1 first and then version 2, and the other half in reverse order. This was done to counter-balance the carryover effects. Snacks were made available for the participants, in case they were prompted and were hungry. The participants were unaware of the possibility of eating snacks, to prevent disturbing the interaction with the robot. Otherwise the subjects would have been primed for eating, which would have biased the results and hide the effect of the robotic interaction.
30 +For the experiment, we used a within-subject design. All of the participants interacted with both versions of the robot, with half of the participants interacting with version 1 first and then version 2, and the other half in reverse order. This was done to counter-balance the carryover effects. Snacks were made available for the participants, in case they were prompted and were hungry. They were fully aware of them and some of the questionnaire prompts might have given them an idea of what our experiment is about (or at least that it's related to food), which might have skewed our results.
31 31  
32 32  == Tasks ==
33 33  
34 -The participant interacted with the robot, which was programmed to engage in a lunch discourse. Two versions were implemented: the first version asks basic questions about mealtime, mostly acting as a reminder for the PwD to have lunch (basically an alarm clock). The second is our original implementation of it with the more sophisticated discourse and music.
34 +The participant interacted with the robot, which was programmed to engage in a lunch discourse. Two versions were implemented: the first version (simple interaction) asks basic questions about mealtime, mostly acting as a reminder for the PwD to have lunch (basically an alarm clock). The second (advanced interaction) is our original implementation of it with the more sophisticated discourse and music.
35 35  
36 36  == Measures ==
37 37  
... ... @@ -41,57 +41,36 @@
41 41  
42 42  The procedure was conducted as follows:
43 43  
44 -1. Welcome participant and explain what they are going to be doing.
44 +1. Welcome participants and explain what they are going to be doing.
45 45  1. Have them sign the permission form.
46 -1. Complete questionnaire 1 regarding their emotional state (control).
47 -1. Have an interaction with version A of the robot.
46 +1. Complete questionnaire 1 regarding their emotional state and hunger scale (control).
47 +1. Have interaction with version A of the robot.
48 48  1. Complete questionnaire 2 (extended version).
49 49  1. Have a short interview during downtime (prepared questions).
50 -1. Have an interaction with version B of the robot.
50 +1. Have interaction with version B of the robot.
51 51  1. Complete questionnaire 3 (extended version).
52 52  1. Have a short interview during downtime (prepared questions).
53 53  
54 -We used the "Wizard of Oz" method for recognizing agreement and disagreement, to make sure that the whole process did not depend on voice recognition being good enough. In practice, this meant that someone was pressing "y" and "n" on the keyboard according to the participants answers in a place the participant did not see, such as behind them.
54 +We used the "Wizard of Oz" method for differentiating agreement and disagreement, to make sure that the whole process did not depend on voice recognition being good enough, and to have an overall smoother interaction. In practice, this meant that someone was pressing "y" and "n" on the keyboard according to the participants' answers, in a place the participant did not see, such as behind them. The robot's responses were hardcoded, with a few different branches available to take into account the variety of answers the participants would give. The only issue encountered was some connectivity delays at times, which only slightly affected a few of the interactions.
55 55  
56 56  == Material ==
57 57  
58 -For the experiments, we used the NAO robot platform, and a laptop to control it with. The participants completed the questionnaires on their phones by scanning a QR code. The questionnaires are a combination of questions regarding the emotional state of the participants, their interaction with the robot, and the music included in the interaction. Stroopwafels and water in a clean cup were made available to see and measure how much people ate.
58 +For the experiments, we used the NAO robot platform, and a laptop to control it. The participants completed the questionnaires on their phones by scanning a QR code. The questionnaires are a combination of questions regarding the emotional state of the participants, their hunger levels, their interaction with the robot, and the music included in the interaction. Stroopwafels and water in a clean cup were made available to see and measure how much people ate.
59 59  
60 -Below are listed the contents of the three questionnares:
60 +During the experiments, four different types of questions were given to the participants, in addition to the Consent Form and Disclaimers they had to sign in the beginning. The four sections were:
61 61  
62 -Questionnare 1:
62 +1. 8 questions from the [[EVEA>>https://www.ucm.es/data/cont/docs/39-2013-04-19-EVEA%20-%20Datasheet.pdf]] questionnaire for mood assessment
63 +1. 4 questions from the [[Godspeed>>https://www.bartneck.de/2008/03/11/the-godspeed-questionnaire-series/]] questionnaire to assess the pleasantness and intelligence of the robot
64 +1. 3 hunger and food-related questions of our own, to assess if they eat before or during the interaction (5-point Likert scale)
65 +1. 2 music-related questions of our own, to measure how much they enjoyed the music and what was its effect (5-point Likert scale)
63 63  
64 -* Consent Form and Disclaimers
65 -* Control for robot version A
66 -** 8 questions from the [[EVEA>>https://www.ucm.es/data/cont/docs/39-2013-04-19-EVEA%20-%20Datasheet.pdf]] questionnaire
67 -** 4 questions from the [[Godspeed>>https://www.bartneck.de/2008/03/11/the-godspeed-questionnaire-series/]] questionnaire
67 +Before the first interaction, the participants were asked to respond to sections 1. and 3., while right after each interaction, they were asked to respond to all four sections, with the music section only present after the advanced interaction. The full questionnaire given to the participants can be found [[here >> attach:NAO_evaluation.docx ]].
68 68  
69 -Questionnare 2:
70 -
71 -* Questions about robot version A
72 -** 3 food-related questions of our own (5-point Likert scale)
73 -** 2 music-related questions of our own (5-point Likert scale)
74 -* Control for robot version B
75 -** 8 questions from the [[EVEA>>https://www.ucm.es/data/cont/docs/39-2013-04-19-EVEA%20-%20Datasheet.pdf]] questionnaire
76 -** 4 questions from the [[Godspeed>>https://www.bartneck.de/2008/03/11/the-godspeed-questionnaire-series/]] questionnaire
77 -
78 -Questionnare 3:
79 -
80 -* Questions about robot version B
81 -** 3 food-related questions of our own (5-point Likert scale)
82 -** 2 music-related questions of our own (5-point Likert scale)
83 -
84 84  == Practicalities ==
85 85  
86 -Before the experiment we:
71 +For actually performing the experiments, there were multiple tasks that had to be performed beforehand.
72 +We first did a practice round by ourselves, which we filmed to have a controlled performance and to be able to give an example of the experiment if needed. For the time that the experiments were going to take place, we first checked our own availability, so we would have at least one person controlling the robot and another interacting with the participant, explaining everything and keeping notes. Afterward, we contacted people from the rest of the groups and friends of ours and decided on a schedule. Then, we had to book the lab, so we accounted for 20 minutes for each participant, as calculated by the test runs we did ourselves, considering that they were going to be some delays. The last step was to buy some stroopwafels and prepare the lab on the day of the experiments.
87 87  
88 -* did a practice round by ourselves
89 -** This was filmed to have a controlled performance to give an example of the experiment if needed
90 -* contacted other groups and decide on scheduling
91 -** Each participant was booked a 20 min slot
92 -* reserved the lab
93 -* bought the stroopwafels
94 -
95 95  = Results =
96 96  
97 97  The results were gathered from 19 personnel, all of whom interacted first with one version of the robot and then the other. 10 of the participants interacted first with the simple version, nine having their first interaction with the advanced version.
Icon NAO_evaluation.docx
Author
... ... @@ -1,0 +1,1 @@
1 +XWiki.SKostakonti
Size
... ... @@ -1,0 +1,1 @@
1 +21.3 KB
Content Icon