Evaluation prototype

Gardening activity

Overview

The prototype used for the evaluation is fully autonomous and does not require any external action except for manually hardcoding data before the first session and then starting the behaviour at the beginning of each session. In particular, the whole activity was done in autonomy by the participants, without any external action.

The evaluation prototype is a concrete example of some use cases merged into a bigger behaviour.

At the beginning, Pepper reminds the participant that they wanted to do home gardening. (This can be for instance a calendar reminder everyday at 17:00. For the evaluation, we obviously did not implement any time checking function, as we wanted the activity to start as soon as the participant said they were ready.)
If the participant agrees, Pepper starts with the first step, asks the participant whether they did it, then waits for the participant to declare that they have completed the step.
The participant may also ask Pepper to repeat, or say they did not succeed. In the first case, Pepper will repeat the last step. In the second case, Pepper will say it is not a problem, and ask the participant if they want to start over, simply redo last step, or completely stop the activity.
In case Pepper does not understand an answer from the participant, she simply declares so and asks the question again.
When the participant finishes the activity, Pepper congratulates them.

Step visualization

Below is the breakdown of the gardening activity, with appropriate visualization for each step in the process. Here we can witness Pepper assisting of the group members with the activity. With these images we can see the steps being performed, and the facial reaction of the participant while performing the activity.

Step 1

The first step consists of Pepper asking the PwD to locate the pot, soil and seeds. 

Step 2

The second step consists of Pepper instructing the PwD to fill the pot with soil until it is almost full.

Step 3

The third step consists of Pepper asking the PwD to put a seed inside the soil, and push it in with their finger. 

Step 4

The fourth step consists of Pepper asking the PwD to plant 3 more seeds in the soil.

Step 5

The fifth step consists of Pepper instructing the PwD to put just a little bit more soil on top and push it down. 

Step 6

The sixth step consists of Pepper asking the PwD to wipe their hands from the soil.

Step 7

 
The last step consists of Pepper instructing the PwD to water the soil.

Congratulate

 
Finally Pepper congratulates the PwD for successfully completing the activty. 

Final prototype

The final prototype includes more use cases than simply the one from the evaluation.
All the four use cases implemented here are also fully autonomous. They are implemented in a very generic way and are not customized for a specific patient.

In order for the prototype to work properly, two things are still missing.
The first one is the "global controller" that would regulate the different behaviours. For instance, for now we check time independently in each reminder behaviour. A "global controller" would do the checks at a higher level, and then trigger the adapted behaviour based on this.
The second thing missing is the interface for a HCP or a relative to enter the information (calendar, reminders, activity steps, images, etc). For now, all the information is hardcoded. However, for an actual use of the prototype, an interface would exist.

A. Paper plane activity

This is an older version of the combination of calendar and activity breakdown. It is not as developed as the gardening activity for the evaluation. However, it shows that the global process is the same, except for hard-coded information (which would be entered by a HCP or a relative, if we had implemented an interface).

B. Meal reminder

- Play gentle music to gain the attention of the PwD.
- Ask if the PwD already had their meal.
- If the PwD did not have meal
  → remind them to have meals
  → wait 2 minutes and repeat.
- If get confirmation
  → congratulate the PwD and finish with a greeting.

C. Medication reminder

- Play gentle music to gain the attention of the PwD.
- Ask if the PwD already took their medication.
- If the PwD did not take their medication
  → remind them to take their medication of the current time of the day
  → wait 2 minutes and repeat.
- If get confirmation
  → congratulate the PwD

D. Calendar reminder

- Start when the event in inner calendar functions awake.
- Play gentle music to gain the attention of the PwD.
- Wait for music to finish.
- Remind the PwD about the upcoming event in their calendar.
- Wish them good luck for the upcoming event.
This behaviour has unfortunately not been implemented in the handed-in version of our prototype. It is very similar to B. Meal reminder and C. Medication reminder, with slightly different sentences for the conversation with the PwD.