Changes for page Test

Last modified by Mathieu Jung-Muller on 2022/04/04 13:52

From 114.4 to 114.5 From 115.1 to 116.1

From version

114.5

edited by Sneha Lodha
on 2022/04/04 00:06

Change comment: There is no comment for this version

To version

115.1

edited by Sneha Lodha
on 2022/04/04 00:36

Change comment: There is no comment for this version

Raw
Rendered

Summary

Page properties (1 modified, 0 added, 0 removed)

Details

Page properties

Content

@@ -330,11 +330,18 @@
  This confirms that PwDs can potentially benefit from a boost of energy from interacting with Pepper and, if the activity is enjoyable, a general improvement in contentness as well.
--=== System assessment ===
++=== System Assessment ===
--[[System assessment, first subset of questions, interactive version>>https://pietro99.github.io/SCE/graphs/first_questionnaire_1.html]]
++**Task Guidance Assessment**
++[[System assessment, task guidance questions, interactive version>>https://pietro99.github.io/SCE/graphs/first_questionnaire_1.html]]
++The results of the system questionnaire will be divided into four different group with similar questions in each group. This is done to improve the readability of the results and provide a more indepth analysis of various aspects of the system setup.
++
  [[image:group1.svg]]
++Figure 2: Graphical representation of results for task guidance subset of the system assessment, with results shown for people who like vs. dislike gardening, along with the average of the sample.
++
++The first group, namely task guidance assessment, contains questions aimed at measuring how easy and pleasant was the activity when being guided by Pepper. The responses for this group are around the slightly agree line, a bit higher for the "Pepper was easy to understand" statement and a bit lower for the "I enjoyed the task more than if I had had to do it alone".
++
  {{html}}
  <!DOCTYPE html>
  <html>
@@ -359,12 +359,11 @@
  </head>
  <body>
--<h5>Wilcoxon Rank-Sum test</h5>
  <h6><i>H0</i>: The distribution of answers from people who like gardening and people who do not like gardening is the same.</h6>
  <table>
    <tr>
--    <th>test results </th>
++    <th><i>Wilcoxon Rank-Sum results</i></th>
      <th>I would have known how to do the whole task without Pepper.</th>
      <th>I think Pepper made the task easier for me.</th>
      <th>I enjoyed the task more than if I had had to do it alone.</th>
@@ -394,12 +394,16 @@
  </html>
  {{/html}}
--In this first system assessment graph, it is shown that participants slightly agree that Pepper made the task easier for them, and generally agree that she was easy to understand.
++Table 3: Results of Wilcoxon Rank-Sum statistical test on task guidance subset of system assessment for people who like vs. dislike gardening
--[[System assessment, second subset of questions, interactive version>>https://pietro99.github.io/SCE/graphs/first_questionnaire_2.html]]
++**Accomplishment and Autonomy Assessment**
++[[System assessment, accomplishment and autonomy subset, interactive version>>https://pietro99.github.io/SCE/graphs/first_questionnaire_2.html]]
  [[image:group2.svg]]
++Figure 3: Graphical representation of results for accomplishment and autonomy subset of the system assessment, with results shown for people who like vs. dislike gardening, along with the average of the sample.
++The second group, namely the accomplishment and autonomy subset has questions concerning the sense of control and accomplishment felt during the task by the participants. The participants on average responded between slightly agree and agree that completing the task was a good accomplishment and that they felt in control while doing it and a bit lower for the statement "I feel like I have accomplished it myself" suggestingthat it is possible for the participants to feel like Pepper is responsible, at least partially, for the accomplishment of the task.
++
  {{html}}
  <!DOCTYPE html>
  <html>
@@ -424,12 +424,11 @@
  </head>
  <body>
--<h5>Wilcoxon Rank-Sum test</h5>
  <h6><i>H0</i>: The distribution of answers from people who like gardening and people who do not like gardening is the same.</h6>
  <table>
    <tr>
--    <th>test results </th>
++    <th><i>Wilcoxon Rank-Sum results </i></th>
      <th>I feel like completing the task was a good accomplishment.</th>
      <th>I feel like I accomplished it myself.</th>
      <th>I felt in control of what I had to do.</th>
@@ -453,13 +453,19 @@
  </html>
  {{/html}}
--The sense of accomplishment is slightly higher for people who like gardening that for those who do not. It is globally around slightly agree.
--An interesting fact to notice is that participants who do not like gardening felt more in control of what they had to do.
++Table 4: Results of Wilcoxon Rank-Sum statistical test on accomplishment and autonomy subset of system assessment for people who like vs. dislike gardening
--[[System assessment, third subset of questions, interactive version>>https://pietro99.github.io/SCE/graphs/first_questionnaire_3.html]]
++The sense of accomplishment is slightly higher for people who like gardening that for those who do not. It is globally around slightly agree. An interesting fact to notice is that participants who do not like gardening felt more in control of what they had to do.
++
++
++**Negative Experiences Assessment**
++[[System assessment, negative experiences subset, interactive version>>https://pietro99.github.io/SCE/graphs/first_questionnaire_3.html]]
  [[image:group3.svg]]
++Figure 4: Graphical representation of results for negative experiences subset of the system assessment, with results shown for people who like vs. dislike gardening, along with the average of the sample.
++The third group, namely negative experiences subset is used to group together questions that measure negative feeling experiences with Pepper. The results show that the participants on average answered between slightly disagree and disagreed. This suggests that Pepper was not frustrating for most people but only for a small fraction of the participants.
++
  {{html}}
  <!DOCTYPE html>
  <html>
@@ -484,12 +484,11 @@
  </head>
  <body>
--<h5>Wilcoxon Rank-Sum test</h5>
  <h6><i>H0</i>: The distribution of answers from people who like gardening and people who do not like gardening is the same.</h6>
  <table>
    <tr>
--    <th>test results </th>
++    <th><i> Wilcoxon Rank-Sum results </i></th>
      <th>I felt annoyed by Pepper.</th>
      <th>I felt frustrated by the task.</th>
      <th>I felt pressured by Pepper.</th>
@@ -513,12 +513,19 @@
  </html>
  {{/html}}
++Table 5: Results of Wilcoxon Rank-Sum statistical test on negative experiences subset of system assessment for people who like vs. dislike gardening
++
++
  The participants globally disagree that the presence of Pepper annoyed, frustrated or pressured them. Those who like gardening actually had a bit more negative feelings regarding the presence of Pepper than those who dislike gardening.
--[[System assessment, fourth subset of questions, interactive version>>https://pietro99.github.io/SCE/graphs/first_questionnaire_4.html]]
++**Social Assessment**
++[[System assessment, social subset, interactive version>>https://pietro99.github.io/SCE/graphs/first_questionnaire_4.html]]
  [[image:group4.svg]]
++Figure 5: Graphical representation of results for social subset of the system assessment, with results shown for people who like vs. dislike gardening, along with the average of the sample.
++The fourth and final group addresses a social subset and is utilized for assessing Pepper's social presence and trustworthiness as felt by the participants. The two statements used are "Pepper cared about helping me" and "I would trust Pepper with more important activities". The responses were on average slightly above the neutral level.
++
  {{html}}
  <!DOCTYPE html>
  <html>
@@ -543,12 +543,11 @@
  </head>
  <body>
--<h5>Wilcoxon Rank-Sum test</h5>
  <h6><i>H0</i>: The distribution of answers from people who like gardening and people who do not like gardening is the same.</h6>
  <table>
    <tr>
--    <th>test results </th>
++    <th><i>Wilcoxon Rank-Sum results</i></th>
      <th>Pepper cared about helping me.</th>
      <th>I would trust Pepper with more important activities.</th>
    </tr>
@@ -569,6 +569,8 @@
  </html>
  {{/html}}
++Table 5: Results of Wilcoxon Rank-Sum statistical test on social subset of system assessment for people who like vs. dislike gardening
++
  This graph shows that the trust in Pepper was highly dependent on whether the participants enjoyed the activity or not.
  == Discussion ==
@@ -579,28 +579,6 @@
  * Scope: The evaluation can be generalized to a larger scope, although with a lot of care, since the evaluation is not fully valid.
  * Ecological validity: The evaluation is partially valid in terms of influence from the environment. The affect assessment questionnaire is the same before the activity and after, with the same environment, so the environment is technically not involved in this. However, the system assessment questionnaire does rely on some elements from the environment.
--**Affect assessment questionnaire**
--
--We analyzed the participants' moods before and after the interaction with Pepper in order to be able to observe positive and negative changes that are caused by the interaction with Pepper. the results showed that, in general, there is a slight increase in positive moods and a slight decrease in negative moods. The Wilcoxon Signed-rank demonstrated that the only statistically significant change happened for contentness and tiredness based on a p-value threshold of 0.05.
--
--It is often the case that PwDs have to perform tasks that are not enjoyable for them such as taking medicines or performing routine activities. In order to analyze the difference in the mood change between people who liked the activity and people who didn't we divided into two groups and performed a Wilcoxon Rank-Sum test. The results show that only the contentness mood shows a statistically significant difference between the two groups.
--This confirms that PwDs can potentially benefit from a boost of energy from interacting with Pepper and, if the activity is enjoyable, a general improvement in contentness as well.
--
--**system questionnaire**
--
--We divided the results from the system questionnaire into 4 separate groups containing similar questions as illustrated by the 4 different graphs above.
--
--The first group contains questions aimed at measuring how easy and pleasant was the activity when being guided by Pepper. The responses for this group are around the slightly agree line, a bit higher for the "Pepper was easy to understand" statement and a bit lower for the "I enjoyed the task more than if I had had to do it alone".
--
--The second group has questions concerning the sense of control and accomplishment felt during the task by the participants. The participants on average responded between slightly agree and agree that completing the task was a good accomplishment and that they felt in control while doing it and a bit lower for the statement "I feel like I have accomplished it myself" suggesting
--that it is possible for the participants to feel like Pepper is responsible, at least partially, for the accomplishment of the task.
--
--The third group is used to group together questions that measure negative experiences with Pepper. The results show that the participants on average answered between slightly disagree and disagreed. This suggests that Pepper was not frustrating for most people but only for a small fraction of the participants.
--
--The fourth and final group is for assessing Pepper's social presence and trustworthiness as felt by the participants. The two statements used are "Pepper cared about helping me" and "I would trust Pepper with more important activities". The responses were on average slightly above the neutral level.
--
--We performed a statistical test for the system questionnaire to see if the difference between the "like gardening" group and the "dislike gardening" group is significant in any of the questions. The results didn't show any statistical significance with the exception of the question "I would trust Pepper with more important activities" which showed that people who liked gardening were more likely to trust Pepper with more important activities.
--
  **observations**
  Despite having on average good results, some participants still found Pepper frustrating or annoying. In order to understand what could have caused that we analyzed the video of the interaction and the feedback from the participants. We observed that when a participant felt frustrated was often due to Pepper's limitation. For example, sometimes Pepper would start listening too late missing part of the participant's answer. It is also common for the participant to say a word that Pepper is not able to understand which can result in the participant being stuck in a loop during the conversation which can be frustrating. We notice that in most of the experiments the experience and the ease of the interaction with Pepper improved as the participant learned how to interact with Pepper.