Changes for page Test
Last modified by Sofia Kostakonti on 2022/04/05 14:08
From version
117.1


edited by Sofia Kostakonti
on 2022/04/04 20:09
on 2022/04/04 20:09
Change comment:
There is no comment for this version
To version
113.1


edited by Sofia Kostakonti
on 2022/04/04 19:41
on 2022/04/04 19:41
Change comment:
Uploaded new attachment "NAO_evaluation.docx", version 1.1
Summary
Details
- Page properties
-
- Content
-
... ... @@ -27,7 +27,7 @@ 27 27 28 28 == Experimental design == 29 29 30 -For the experiment, we used a within-subject design. All of the participants interacted with both versions of the robot, with half of the participants interacting with version 1 first and then version 2, and the other half in reverse order. This was done to counter-balance the carryover effects. Snacks were made available for the participants, in case they were prompted and were hungry. The ywerefullyaware of themand someofthequestionnaire promptsmighthavegiven themanideafwhatour experimentisabout(or atleastthatit'srelatedtofood), whichmighthave skewedourresults.30 +For the experiment, we used a within-subject design. All of the participants interacted with both versions of the robot, with half of the participants interacting with version 1 first and then version 2, and the other half in reverse order. This was done to counter-balance the carryover effects. Snacks were made available for the participants, in case they were prompted and were hungry. The participants were unaware of the possibility of eating snacks, to prevent disturbing the interaction with the robot. Otherwise the subjects would have been primed for eating, which would have biased the results and hide the effect of the robotic interaction. 31 31 32 32 == Tasks == 33 33 ... ... @@ -41,36 +41,57 @@ 41 41 42 42 The procedure was conducted as follows: 43 43 44 -1. Welcome participant sand explain what they are going to be doing.44 +1. Welcome participant and explain what they are going to be doing. 45 45 1. Have them sign the permission form. 46 -1. Complete questionnaire 1 regarding their emotional state and hunger scale(control).47 -1. Have interaction with version A of the robot. 46 +1. Complete questionnaire 1 regarding their emotional state (control). 47 +1. Have an interaction with version A of the robot. 48 48 1. Complete questionnaire 2 (extended version). 49 49 1. Have a short interview during downtime (prepared questions). 50 -1. Have interaction with version B of the robot. 50 +1. Have an interaction with version B of the robot. 51 51 1. Complete questionnaire 3 (extended version). 52 52 1. Have a short interview during downtime (prepared questions). 53 53 54 -We used the "Wizard of Oz" method for differentiating agreement and disagreement, to make sure that the whole process did not depend on voice recognition being good enough, and to have an overall smoother interaction. In practice, this meant that someone was pressing "y" and "n" on the keyboard according to the participants'answers,in a place the participant did not see, such as behind them.The robot's responses were hardcoded, with a few different branches available to take into account the variety of answers the participants would give. The only issue encountered was some connectivity delays at times, which only slightly affected a few of the interactions.54 +We used the "Wizard of Oz" method for recognizing agreement and disagreement, to make sure that the whole process did not depend on voice recognition being good enough. In practice, this meant that someone was pressing "y" and "n" on the keyboard according to the participants answers in a place the participant did not see, such as behind them. 55 55 56 56 == Material == 57 57 58 -For the experiments, we used the NAO robot platform, and a laptop to control it. The participants completed the questionnaires on their phones by scanning a QR code. The questionnaires are a combination of questions regarding the emotional state of the participants, their hunger levels, their interaction with the robot, and the music included in the interaction. Stroopwafels and water in a clean cup were made available to see and measure how much people ate.58 +For the experiments, we used the NAO robot platform, and a laptop to control it with. The participants completed the questionnaires on their phones by scanning a QR code. The questionnaires are a combination of questions regarding the emotional state of the participants, their interaction with the robot, and the music included in the interaction. Stroopwafels and water in a clean cup were made available to see and measure how much people ate. 59 59 60 - During theexperiments, fourdifferenttypes of questionsweregivento theparticipants, in additiono theConsentFormand Disclaimerstheyhadto sign in thebeginning.The four sections were:60 +Below are listed the contents of the three questionnares: 61 61 62 -1. 8 questions from the [[EVEA>>https://www.ucm.es/data/cont/docs/39-2013-04-19-EVEA%20-%20Datasheet.pdf]] questionnaire for mood assessment 63 -1. 4 questions from the [[Godspeed>>https://www.bartneck.de/2008/03/11/the-godspeed-questionnaire-series/]] questionnaire to assess the pleasantness and intelligence of the robot 64 -1. 3 hunger and food-related questions of our own, to assess if they eat before or during the interaction (5-point Likert scale) 65 -1. 2 music-related questions of our own, to measure how much they enjoyed the music and what was its effect (5-point Likert scale) 62 +Questionnare 1: 66 66 67 -Before the first interaction, the participants were asked to respond to sections 1. and 3., while right after each interaction, they were asked to respond to all four sections, with the music section only present after the advanced interaction. 64 +* Consent Form and Disclaimers 65 +* Control for robot version A 66 +** 8 questions from the [[EVEA>>https://www.ucm.es/data/cont/docs/39-2013-04-19-EVEA%20-%20Datasheet.pdf]] questionnaire 67 +** 4 questions from the [[Godspeed>>https://www.bartneck.de/2008/03/11/the-godspeed-questionnaire-series/]] questionnaire 68 68 69 +Questionnare 2: 70 + 71 +* Questions about robot version A 72 +** 3 food-related questions of our own (5-point Likert scale) 73 +** 2 music-related questions of our own (5-point Likert scale) 74 +* Control for robot version B 75 +** 8 questions from the [[EVEA>>https://www.ucm.es/data/cont/docs/39-2013-04-19-EVEA%20-%20Datasheet.pdf]] questionnaire 76 +** 4 questions from the [[Godspeed>>https://www.bartneck.de/2008/03/11/the-godspeed-questionnaire-series/]] questionnaire 77 + 78 +Questionnare 3: 79 + 80 +* Questions about robot version B 81 +** 3 food-related questions of our own (5-point Likert scale) 82 +** 2 music-related questions of our own (5-point Likert scale) 83 + 69 69 == Practicalities == 70 70 71 -For actually performing the experiments, there were multiple tasks that had to be performed beforehand. 72 -We first did a practice round by ourselves, which we filmed to have a controlled performance and to be able to give an example of the experiment if needed. For the time that the experiments were going to take place, we first checked our own availability, so we would have at least one person controlling the robot and another interacting with the participant, explaining everything and keeping notes. Afterward, we contacted people from the rest of the groups and friends of ours and decided on a schedule. Then, we had to book the lab, so we accounted for 20 minutes for each participant, as calculated by the test runs we did ourselves, considering that they were going to be some delays. The last step was to buy some stroopwafels and prepare the lab on the day of the experiments. 86 +Before the experiment we: 73 73 88 +* did a practice round by ourselves 89 +** This was filmed to have a controlled performance to give an example of the experiment if needed 90 +* contacted other groups and decide on scheduling 91 +** Each participant was booked a 20 min slot 92 +* reserved the lab 93 +* bought the stroopwafels 94 + 74 74 = Results = 75 75 76 76 The results were gathered from 19 personnel, all of whom interacted first with one version of the robot and then the other. 10 of the participants interacted first with the simple version, nine having their first interaction with the advanced version.