Skip to Content

Changes for page Humanoid Robot

Last modified by Varun Singh on 2023/02/28 18:32
From version 1.1
edited by Ruud de Jong
on 2023/02/13 12:00
Change comment: Imported from XAR
To version 3.1
edited by Varun Singh
on 2023/02/28 16:51
Change comment: There is no comment for this version

Summary

Details

Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -xwiki:XWiki.RuuddeJong
1 +XWiki.varunsingh3000
Content
... ... @@ -1,0 +1,92 @@
1 +Our main goal for this project was to use art specifically painting as a medium to bring joy and happiness to patients with dementia. Drawing and paintings have been shown to bring happiness (1) and also increase and maintain creativity (2, creativity paper) in the lives of patients with dementia.
2 +
3 += **Robot** =
4 +
5 +We had to make a choice among the 3 robots [[Pepper>>https://www.aldebaran.com/en/pepper]], [[Nao>>https://www.aldebaran.com/en/nao]], and [[Miro>>https://www.miro-e.com/]]. We've chosen to go ahead with Pepper because of a few reasons. One of them is that our robot needs to be interactive in terms of displaying the sample picture which the PwD has to follow and use as inspiration for drawing. Pepper conveniently provides us with a tablet that can accomplish the task of displaying pictures and acting as an interactive user interface. The tablet has been installed in front of its body which is the main reason for choosing Pepper.  
6 +
7 += **Pepper Description** =
8 +
9 +As mentioned above Pepper has a tablet which provides us with graphical user interface capabilities. Along with these it also has additional features and sensors like Speech Recognition, Natural Language Processing functionalities such as being able to understand as natural language being spoken to it. Pepper also has sensors like cameras in its eyes, a distance sensor, audio sensors, sensors in its hand, infrared sensors and sonars along its base. These sensors help it to perform activities like navigation, and human-like interaction with people.
10 +
11 +
12 +(Add image of pepper)
13 +
14 +
15 += **Usage** =
16 +
17 +For our use case, we needed the tablet, the cameras, the audio sensors, navigation capabilities of the robot such as the hand and gyro base movement. However, due to time constraints, we had to work around a lot of the capabilities of the robot to come up with a working prototype for this use case in time. We have defined a flow of the actions that need to happen given these time constraints were removed and all the implementation could be on the robot using its SDK.
18 +
19 +
20 +At a broad level, the main actions where the robot is expected to support the patient during painting are as follows:
21 +
22 +~1. Help the patient choose a style and topic
23 +
24 +2. Have relevant information wherever necessary on the tablet
25 +
26 +3. Provide valuable suggestions and help related to painting
27 +
28 +4. If requested play some calming music to soothe the patient during the painting process
29 +
30 +5. Capture a snapshot of the painting once its done, save it on its memory and later send it to the family member
31 +
32 +
33 +These actions have been elaborated on below. Due to feasibility and time constraints, most of the functionalities of the robot have to be worked around using a wizard of oz approach. For this reason, we have two different sets of the flow of actions, one being the actions that in an ideal scenario should have happened using the Pepper robot and another which is the actual step of actions we took. For the envisioned actions we are confident that all of them are technically possible using the Pepper robot and its SDK given enough time and effort. Our reasoning for this is that we are more interested in evaluating and examining the effect of such a prototype even if it's at a small-scale project level and at least for now not get bothered by implementation errors and problems.
34 +
35 +== **Actions envisaged** ==
36 +
37 +Note: Almost every interaction will take place both using audio i.e. the robot's voice and speech recognition capabilities and visual i.e. the tablet interface.
38 +
39 +Assumptions: Have the painting apparatus ready along with all the necessary equipment in the recreational room.
40 +
41 +1) During the patient's entertainment time in the recreational room, approach them and ask using audio whether they would be interested in painting something today. Since a lot of people at that age can be hearing impaired, the same question would also be asked by displaying it on the tablet interface.
42 +
43 +2) If the PwD replies yes, further painting interaction will take place, otherwise, the painting interaction will not and the patient is free to do what they wish.
44 +
45 +3) Assuming the patient said yes, then they are asked the painting style they would like to go ahead with it for their drawing.
46 +
47 +4) Once they choose a style certain information and instructions regarding the style is given to help them to replicate the same style as closely as possible.
48 +
49 +5) Next, the topic that they would like to draw is asked, this can be something personal like their past memories which can be retrieved using the memory data stored in the robot for each patient regarding their past which is provided by their family members. They could also select from a range of preloaded topics or draw whatever they wish to.
50 +
51 +6) While the patient is drawing and painting, the robot will give specific instructions regarding the style they chose and also offer some general suggestions related to painting to help the patient.
52 +
53 +7) If the patient desires they could also use a button on the tablet interface to play some calming white music but if they don't like it or find it annoying they can turn it off.
54 +
55 +8) Finally when the patient is done with the painting, they could tell the robot or just use the tablet to indicate that they are done.
56 +
57 +9) The robot will then use its stereo eye cameras to take a picture of the painting and send it to the family member so that during their next visit they can also talk about the painting which should strengthen the interaction and relationship between the family member and the PwD.
58 +
59 +== **Actions actually done** ==
60 +
61 +In this section, we are going to write the steps we actually took while implementing the prototype. This is going to be an interactive process and will get changed as and when we get further into the actual implementation of the project.
62 +
63 +Note: Since programming and using the Pepper robot's SDK is tedious and time-consuming, for the functionalities of using the robot's tablet interface and cameras, we are going to work around that. For the table interface, we would create a web interface on a separate tablet with most of the features mentioned above that are possible. For the cameras, we would right now use a wizard of oz approach and have a person imitate that functionality or assume it to be taken care of. We are also not going to be using the navigational capabilities of the robot since it's not critical for the painting task.
64 +
65 +1) During the patient's entertainment time in the recreational room, ask using audio whether they would be interested in painting something today. Since a lot of people at that age can be hearing impaired, the same question would also be asked by displaying it on the tablet interface.
66 +
67 +2) If the PwD replies yes, further painting interaction will take place, otherwise, the painting interaction will not and the patient is free to do what they wish.
68 +
69 +3) Assuming the patient said yes, then they are asked the painting style they would like to go ahead with it for their drawing.
70 +
71 +4) Once they choose a style certain information and instructions regarding the style is given to help them to replicate the same style as closely as possible.
72 +
73 +5) Next, the topic that they would like to draw is asked, this can be something personal like their past memories which can be retrieved using the memory data stored in the robot for each patient regarding their past which is provided by their family members. They could also select from a range of preloaded topics or draw whatever they wish to.
74 +
75 +6) While the patient is drawing and painting, the robot will give specific instructions regarding the style they chose and also offer some general suggestions related to painting to help the patient.
76 +
77 +7) If the patient desires they could also use a button on the tablet interface to play some calming white music but if they don't like it or find it annoying they can turn it off.
78 +
79 +8) Finally when the patient is done with the painting, they could tell the robot or just use the tablet to indicate that they are done.
80 +
81 +9) This could be assumed to be taken care of and implemented in the ideal version of this system or a person who is acting at certain points as the robot just use the table to take a picture of the painting. This image could potentially be stored in the robot's memory if feasible.
82 +
83 +(Need to discuss regarding the memory parts, depending on how complicated the implementation actually is, this might have to be dropped)
84 +
85 +Refs:
86 +
87 +[1]: [[https:~~/~~/journals.sagepub.com/doi/pdf/10.1177/1471301211421090>>https://journals.sagepub.com/doi/pdf/10.1177/1471301211421090]]
88 +
89 +[2]: [[https:~~/~~/www.karger.com/Article/Abstract/89827>>https://www.karger.com/Article/Abstract/89827]]
90 +
91 +
92 +