Changes for page Test

Last modified by Clara Stiller on 2022/04/05 13:44

From 51.1 to 52.1 From 52.3 to 53.1

From version

52.1

edited by Clara Stiller
on 2022/03/27 15:48

Change comment: There is no comment for this version

To version

52.3

edited by Sayak Mukherjee
on 2022/04/02 02:40

Change comment: There is no comment for this version

Raw
Rendered

Summary

Page properties (2 modified, 0 added, 0 removed)

Details

Page properties

Author

@@ -1,1 +1,1 @@
--XWiki.ClaraStiller
++XWiki.SayakMukherjee

Content

@@ -1,5 +1,5 @@
--The best way to test our prototype would be a study with persons with dementia. Still, testing the robot in a real environment would be very time-consuming, because it is not predictable if and when people with dementia start wandering. That is out of scoop for our project.
--However, we want to get a first impression of how realistic and convincing the provided dialogues and suggested activities are. In a small study with students, who play the role of having dementia, we are observing the interaction with the robot and want to find out how effective it is in preventing people from wandering.
++The best way to test our prototype would be a study with persons with dementia. However, testing the robot in a real environment would be very time-consuming, because it is not predictable if and when people with dementia start wandering. Hence it is out of the scope of this project.
++Nevertheless, we wanted to get a first impression of how realistic and convincing the provided dialogues and suggested activities are. In a small study with students, who play the role of having dementia, we are observing the interaction with the robot with an aim to find out how effective it is in preventing people from wandering.
  = Problem statement and research questions =
@@ -101,7 +101,8 @@
  = Results =
--=== Comparison between intelligent (cond. 1) and less intelligent (cond. 2) prototype ===
++{{html}}
++<!--=== Comparison between intelligent (cond. 1) and less intelligent (cond. 2) prototype ===
  {{html}}
  <img src="/xwiki/wiki/sce2022group05/download/Test/WebHome/Stay_inside.svg" width="500" height="270" />
@@ -138,9 +138,8 @@
  ==== Condition 2 - less intelligent prototype: ====
--Participants assigned to condition 2 weren't convinced to leave. We saw, that most of them tried to continue talking to pepper when it raises its arm to block the door, even though it didn't listen. They were surprised by peppers reaction and asked for a reason why they are not allowed to leave. In order to have a natural conversation flow, the robot should provide an explanation for each scenario that tells why the person is not allowed to leave. This confirms that our approach, to give reason to stay inside, might be helpful to convince PwD to stay inside.
++Participants assigned to condition 2 weren't convinced to leave. We saw, that most of them tried to continue talking to pepper when it raises its arm to block the door, even though it didn't listen. They were surprised by peppers reaction and asked for a reason why they are not allowed to leave. In order to have a natural conversation flow, the robot should provide an explanation for each scenario that tells why the person is not allowed to leave. This confirms that our approach, to give reason to stay inside, might be helpful to convince PwD to stay inside.
--
  === Problems that occurred during the evaluation ===
 . lots of difficulties with speech recognition:
@@ -154,21 +154,135 @@
  One of the most frequent and noticeable reactions from participants was **confusion**. This feeling was caused by two main factors:
  misunderstandings from speech recognition which leads to unsuitable answers from pepper, as well as the unsuitable environment and setting of our evaluation.
  The reasons for failure in speech recognition are listed above. An unsuitable answer can e.g. be an argument to stay inside, that doesn't fit the participant's reason to leave. Also, some people told in a long sentence that they don't like the provided activity and still want to leave. If the speech recognition fails in this case and pepper understood you would like to do the activity, it seems like it encourages you to leave, instead of doing the activity. This leads to the total opposite of our intention.
--Furthermore, we found out, that our prototype doesn't fit in the environment of the lab. We encourage the participant to do some activities, that they can't do in the lab environment (go to the living room, have a coffee or do a puzzle). If the robot tells asks you if you want to do the activity, most people don't know how to react and are insecure about how to answer. Participants "freeze" in front of the robot or just left the room.
++Furthermore, we found out, that our prototype doesn't fit in the environment of the lab. We encourage the participant to do some activities, that they can't do in the lab environment (go to the living room, have a coffee or do a puzzle). If the robot tells asks you if you want to do the activity, most people don't know how to react and are insecure about how to answer. Participants "freeze" in front of the robot or just left the room. -->
++{{/html}}
++1. What percentage of people are prevented from going out unsupervised? (Quantitative) (CL01, CL05)
++1. How does the interaction change the participant's mood? (CL02)
++1. Can the robot respond appropriately to the participant's intention? (CL03)
++1. How do the participants react to the music? (CL04)
++1. Does the activity that the robot suggests prevent people from wandering/ leaving? (CL06)
++1. Can pepper identify and catch the attention of the PwD?
++=== RQ1: Are people convinced not to go out unsupervised? ===
++{{html}}
++<table style="width: 100%">
++<tr>
++<td style="width: 50%">
++<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/Storyboard_1.png?height=750&rev=1.1" />
++</td>
++<td>
++Comment on the graph
++</td>
++</tr>
++</table>
++{{/html}}
--= Discussion =
++=== RQ2: How does the interaction change the participant's mood? ===
++{{html}}
++<table style="width: 100%">
++<tr>
++<td style="width: 50%">
++<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/Storyboard_1.png?height=750&rev=1.1" />
++</td>
++<td>
++Comment on the graph
++</td>
++</tr>
++</table>
++{{/html}}
++=== RQ3: Can the robot respond appropriately to the participant's intention? ===
++{{html}}
++<table style="width: 100%">
++<tr>
++<td style="width: 50%">
++<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/Storyboard_1.png?height=750&rev=1.1" />
++</td>
++<td>
++Comment on the graph
++</td>
++</tr>
++</table>
++{{/html}}
++=== RQ4: How do the participants react to the music? ===
++{{html}}
++<table style="width: 100%">
++<tr>
++<td style="width: 50%">
++<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/Storyboard_1.png?height=750&rev=1.1" />
++</td>
++<td>
++Comment on the graph
++</td>
++</tr>
++</table>
++{{/html}}
++=== RQ5: Does the activity that the robot suggests prevent people from wandering/ leaving? ===
++{{html}}
++<table style="width: 100%">
++<tr>
++<td style="width: 50%">
++<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/Storyboard_1.png?height=750&rev=1.1" />
++</td>
++<td>
++Comment on the graph
++</td>
++</tr>
++</table>
++{{/html}}
++=== RQ6: Can pepper identify and catch the attention of the PwD? ===
++{{html}}
++<table style="width: 100%">
++<tr>
++<td style="width: 50%">
++<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/Storyboard_1.png?height=750&rev=1.1" />
++</td>
++<td>
++Comment on the graph
++</td>
++</tr>
++</table>
++{{/html}}
++=== Reliabity Scores ===
++{{html}}
++<table style="width: 100%">
++<tr>
++<td style="width: 50%">
++<img src="/xwiki/wiki/sce2022group05/download/Foundation/Operational%20Demands/Personas/WebHome/Storyboard_1.png?height=750&rev=1.1" />
++</td>
++<td>
++Comment on the graph
++</td>
++</tr>
++</table>
++{{/html}}
++= Limitation=
++
++* **Lab Environment**: The lab environment is different from a care home, the participants found it difficult to process the suggestions made by Pepper. For example, if Pepper asked someone to visit the living room, it created confusion among the participants regarding their next action.
++
++* **Role-Playing**: Participants for the experiment are not actual patients suffering from dementia. Hence it is naturally difficult for them to enact the situations and replicate the mental state of an actual person suffering from dementia.
++
++* **Speech Recognition**: The speech recognition module inside Pepper is not perfect. Therefore, in certain cases, Pepper misinterpreted words spoken by the participants and triggered an erroneous dialogue flow. The problems commonly occurred with words that sound similar such as "work" and "walk". Moreover, there are some additional hardware limitations that hampered the efficiency of the speech recognition system. One prominent issue is that the microphone within Pepper is only active when the speaker is turned off. A blue light in the eye of Pepper indicated the operation of the microphone. Since most of the participants are not used to interacting with Pepper found it difficult to keep this limitation in mind while trying to have a natural conversation.
++
++* **Face Detection**: The face recognition module within Pepper is also rudimentary in nature. It can not detect half faces are when participants approach from the side. Adding to the problem, the lighting condition in the lab was not sufficient for the reliable functioning of the face recognition module. Hence Pepper failed to notice the participant in some cases and did not start the dialogue flow.
++
  = Conclusions =
++* People who liked the activity tend to stay in
++* People who knew the music found it more fitting
++* People are more convinced to stay in with the intelligent prototype
++* Cannot conclude whether moods were improved
++* Need to experiment with the actual target user group to derive on concrete conclusion
++* Experiment with personalization
--just some notes...
++= Future Work =
++* **Personalisation**: Personalize music, and activity preferences according to the person interacting with Pepper.
++* **Robot Collaboration**: Collaborate with other robots such as Miro to assist a person with dementia while going for a walk instead of the caretaker.
++* **Recognise Person**: For a personalised experience, it is essential that Pepper is able to identify each person based on an internal database.
++* **Fine Tune Speech Recognition**: Improvements are necessary for the speech recognition module before the actual deployment of the project in a care home. Additionally, support for multiple languages can be considered to engage with non-English speaking people.
--* Evaluation is not significant enough to answer our main question/ goal, due to many issues with speech reco and misunderstandings
--* difference in effectiveness between cond 1 and cond 2, but not in all cases
--* Confusion + waiting for the robot to listen -> bad for PwD, because they are already confused and might not remember to wait for the eyes to turn blue