Hrishita - Self Reflection

Week 1: Introduction to Socio-Cognitive Engineering

Human-Centred design:

We have to consider certain factors when designing and/or assessing a human-centred design such as the extent of usability of the system by the target users, and how effective and efficient the system is in achieving the target goal. When designing the system we should also keep in mind the context the system will be used in and should also ensure the users have a pleasant experience when using the system. 

Music in robot-guided activities:

Why music? It evokes emotion, certain songs can trigger memories leading to reminiscence and also helps to connect people with similar music tastes. It also creates a pleasant environment for exercise. 

Talked further in detail in next week's guest lecture.

ReJAM:

Targeted for elderly residents of Pieter Van Foreest suffering from dementia

The ReJAM project is designed to stimulate the patients socially, cognitively, affectively, and physically through music-related activities

Some examples of activities: 

- Playing the patient's favourite music and asking them if it reminds them of something (reminiscence)

- Asking the patient to follow the robot's movement to dance along with it (physical rehab)

- Playing music bingo (memory practice)

The robot would also act as a social actor and be a companion to the elderly patients by chatting with them, singing songs together, playing games, etc

Learning with Charlie:

Targeted for children suffering from Type 1 diabetes and their parents and caregivers. 

The PAL system consists of a robot Charlie that kids can interact with when they visit the hospital for checkups and can also interact with them virtually via an app in which they can play games with an avatar of the robot. 

Through these games, the robot is used to inculcate a healthy lifestyle suitable for children suffering from diabetes so that they remember to take their measurements and medicines in time and are motivated to exercise regularly to regulate their blood sugar levels. 

Lab Activity:

Formed teams and discussed a possible use case for developing a system of robot intervention for aiding a Person with Dementia (PwD). We learnt about the different robots available for the project - Nao, Pepper, and Miro.

My team and I decided to focus on mealtime, especially during dinner when a PwD at the moderate stage of dementia suffers from confusion, anxiety and irritability due to Sundown Syndrome. We created three personas to develop our use-case scenario - Georgina (the PwD), Eleana (formal caretaker), and Sam (family member). After discussions within the team, we decided that the Pepper robot would be the best suited for our use case as it can narrate stories and display pictures on the screen attached to its torso. While both Nao and Pepper are humanoid robots capable of holding conversations, we decided to stick with Pepper as our use case did not require too much movement by the robot, and the height of the Pepper robot was also better suited for this use case such that it could stand in front or around the PwD while narrating the story.

We received positive feedback for our chosen problem scenario so we decided to conduct further research into existing literature about the use case for better understanding and consequent system design. 

Week 2: Dementia, Memory and Music as an Intervention

Cognitive change is natural:

Change in cognitive abilities is natural with age however the change is not a decline for all kinds of cognitive abilities. Some abilities such as processing speed and memory do decline however word knowledge such as vocabulary tends to increase with age as we are exposed to more information with time.

The rate of cognitive decline is largely dependent on an individual's lifestyle. For instance, a history of diabetes or depression, low education level, lack of exercise and balanced nutrition, smoking, etc can all contribute to a quicker cognitive decline.

Cognitive reserve:

The brain changes structurally and functionally with time. However, there is an inherent quality to an individual's brain, known as Cognitive reserve that can explain how susceptible the individual would be to functional impairment in case of some disease/accident. The more the cognitive reserve of an individual, the longer a neurological disease/accident takes to cause serious functional impairment in the brain. 

The cognitive reserve can only be increased with a lifetime of healthy lifestyle choices such as a better education level, proper exercise and balanced nutrition, regular cognitive engagement, etc.

Dementia:

Common symptoms: Significant decline in 1 or more cognitive domains, the decline is at a stage that it begins to interfere with the individual's daily lifestyle. The individual might also show some behavioural disturbance.

There are pharmacological treatments available and being researched into however they are not meant for long term, they only delay the symptoms by some time.

Other measures of interventions - aim at maintaining cognition and regulating mood; the intervention measures are to prevent the individual from feeling "more disabled" than they actually are and maintain some level of self-sufficiency and independence in their life.

Current interventions for maintaining cognition can be split into 3 parts: Stimulation (usually in groups), Training, and Rehabilitation.

Music as an intervention?

The normal human brain uses various pathways/networks of the brain to perceive different aspects of music. The brain is highly engaged when perceiving music hence the great potential for using it in rehabilitation. Music is already used by everyday individuals as a mood regulator in their everyday lives, it also provides a great opportunity for social interaction, and can even get the person moving to the rhythm (potential for movement rehabilitation).

Week 3: PAL

The robot intervention system PAL was aimed at teaching kids with Type 1 diabetes how to become self-sufficient in managing their blood glucose levels (T1DM). The system through games and conversations would remind the kid to check their blood glucose levels and administer their prescribed dosage of insulin regularly, look forward to their meals and exercises and overall learn to cope with T1DM.

Through the project presentation, I was able to learn several aspects of robot intervention designs and how they can be evaluated. 

Collaborative Learning:

The PAL system was based on the theory of collaborative learning which believes that individuals learn better when they actively interact with the information such as through knowledge exchange, sharing their experiences, etc. The educator's role in this kind of learning is to ensure that the learner's experience is within their Zones of Proximal Development (ZPD), i.e the content to be learnt should be in an optimal zone where the difficulty of the content is not too high for the learner's skill level that they are left confused but it should neither be too simple for the learner's skill level that they are bored.  

In PAL they made use of collaborative learning by deploying a robot which would not only play along with the kid but the robot would also adapt the tasks in the game to the kid's learning progress. The developers believed this would motivate the kids to perform the activity as it is tailored to their capabilities and would therefore be fun for them, however, they also noted that some kids may not reach the minimum performance level that they set within the time of evaluation due their relatively long learning curve. Their evaluation results corroborated their claim, and the children chose to play with the adaptive robot more often than the non-adaptive variant. They also noticed when using the adaptive robot, individual patients converted to their personal level of learning over the evaluation period which also corroborated their second claim. In the case of the non-adaptive robot, the kids all converged to a more common learning level with a lower overall mean across the patients. 

Week 4: First Presentation

My teammate and I presented our chosen problem scenario and our plans for the robot intervention. We elaborated on our personas and the issues all our direct stakeholders are facing during Georgina (the PwD)'s mealtime. Georgina is losing interest in her meals, Sam her son feels hesitant to talk to his mother for fear of triggering her anxiety/irritability and Eleana wishes to see her patient happy. Thus we introduce an intervention wherein the robot takes the role of a storyteller and engages all the present direct stakeholders in easy-going and nostalgic conversations around the story it narrates. 

At the end of the presentation, we received some feedback and questions which we then used to improve our design for our implementation.

Week 5: Design Specifications

Task Level Design (Collaboration):

Requirement: What should the robot be doing? This forms the function(s) of the robot.

Use Case: Where is the robot working? When should it intervene? Who are the actors involved? This gives the robot the context

Motivation effect: Why should the robot be performing the action? What are we as designers claiming will be the effect of the robot's action? (Justify with theory and empirical studies)

Team Design Patterns (TDP):

Abstract representation of the Task Level Design

TDP Table consists of the Problem, the Solution Structure (Diagrammatic), the Solution Description (Action Sequence), the Human and Robot requirements, and the Consequences (Effect)

Interaction Level Design (Communication):

How will the robot perform the requirement?

Form a premise based on some literature

For example: In PAL system the premise for the robot to sympathise with the child was that the child will recognize the robot's bodily emotional expressions correctly hence the child will feel like the robot is able to understand them and will like to interact with the robot more and consequently learn more as they interact.

Interaction Design Patterns (IDP): Abstract representation of the Interaction Level Design

Evaluations:

- Evaluations should show that the claims were right; should also check that the implementations are safe (privacy and safety of all the actors involved)

- Formative Evaluations 

• open questions which explore the quality of design and how to improve it
• for our use case, we decided not to use formative evaluations since our evaluations would only be conducted by the person with dementia and in a real-world scenario they may not be able to provide such an analysis

- Summative Evaluations

• Focus on overall effect; Investigate whether people are able to reach the objectives
• Yes/No questions which answer our hypotheses
• We chose summative evaluations as the person with dementia would be able to let us know accurately whether their mood was improved after using the robot or not; also whether they finished their meal or not was another measurement that can easily be taken

- Measurements

• Objective (efficiency and effectiveness) v/s Subjective (user satisfaction) measures
• For our use case we planned on both objective (whether the patient finishes their meal or not - effectiveness) and Subjective (whether the patient's mood improved after using our solution or not) measures
• However, upon further discussions, we realised that objective measurement would not be possible as the evaluation conducted would be in a simulated environment hence the effectiveness could not be reliably measured, hence we decided to evaluate the subjective measurements

- Online Experimentation

• To avoid using paper for our evaluations we decided to host our questionnaires online
• Our first go-to was Google Forms but upon learning it doesn't abide by the regulations of GDPR we switched to Qualtrics for our evaluation forms

Week 6: Implementation and Initial Testings

This week we focused on finalising our system design and implementing our design using the Interactive Robots platform. For our evaluation, we decided on testing our robot in two scenarios - one where it encouraged conversation while narrating a story from the patient's past; the second where it only narrated the story as the patient had their meal. To avoid any bias from the story's content, we decided that the story narrated in both scenarios would be the same but we still would like the patient/caretaker to have some choices when it comes to which story they would like to hear to avoid monotony. 

The team and I then brainstormed on short but nostalgic stories and came up with two stories - a picnic outing with family and a Thanksgiving dinner with family. We fleshed out the story with the dialogues and prompts and then implemented the story flow on the Interactive Robots platform. During the implementation, we realised that using Interactive Robots we couldn't connect to Pepper's tablet so we modified the prompts such that they would appear on the device held by the caretaker. We first tested out the motions and flow of the story on the virtual robot and once we were satisfied we booked a slot for testing it on the Pepper robot.

While testing with Pepper we realised the audio inputs weren't very accurate and often the robot failed to pick up the trigger word so we modified the triggers such that the robot would respond to speech or an alternative touch/remote action such that the flow of conversation is not interrupted during the actual evaluation. 

Week 7: Evaluation

Our hypothesis was that a more interactive i.e conversational robot (experiment scenario) would be better at improving the PwD's mood as well as creating a more immersive and enjoyable storytelling session which would motivate the PwD to finish their meal enthusiastically.

Due to the limited number of participants as well as the limited time available for evaluation, my team and I decided to conduct a within-study for our evaluation. We invited fellow students taking this course as well as other TU delft students to participate in our experiment. Each participant would be first asked to sign a consent form after which we explained to them how the evaluation would be conducted. The participant would first perform one story session with the robot and then report their evaluation through a questionnaire and then perform another story session with the robot followed by the same questionnaire once again. In both scenarios, the participant took the role of the PwD while the roles of the formal caretaker and family member were performed by one of us within the team. 

There were two types of storytelling sessions which the participant had to participate in and evaluate:

1. Experiment scenario, wherein the robot narrated the story and in between asked questions to spark conversations

2. Control Scenario, wherein the robot narrated the story and enacted conversations via voice modulations to portray different characters 

We were able to perform the experiment with 14 participants. Half of the participants started with the control scenario and the other half started with the experiment scenario so that our results and analysis would not be influenced by the carry-over bias. The questionnaire used for evaluation was based on the Godspeed questionnaire which tests perceived anthropomorphism, animacy, safety and the threat of the robot. We modified the questionnaire to also evaluate the mood of the patient (participant) after each story session as well as how much they enjoyed the story.

We conducted a one-tailed paired test (dependent t-test) to test for the statistical significance of our results. All three of our added questions were significant proving that the conversational robots significantly improved the patient's mood in comparison to the non-conversational robot. Some other significant differences between the conversational robot and non-conversational robot were that the patients in general perceived the conversational robot to be more natural and responsible and they also liked the conversational robot more than the non-conversational robot.

Week 8: Final Presentation

My teammates presented our project. They first began with a quick recap of our problem scenario and personas and then moved on to our design scenario elaborating on the theories on which we based our design. Then they explained our experiment and control scenarios, followed by our evaluation procedure and results, and finally wrapped up with our takeaways and limitations. 

We received some feedback on our choice of evaluation questionnaire and statistical analysis which we then added to our XWiki report as part of our critical analysis and takeaway for future projects.