Step 4: Claims
| Topic | Question | Answer |
Measurements | For each positive and negative effect listed in step 3, describe how you could evaluate (measure) whether they actually occur. | - Memory accuracy. To evaluate whether the robot can help increase memory accuracy, we can look at the percentage of times that the patient names relatives and their relationships correctly. - Social connection. To evaluate whether the robot can help promote social connection, we can look at the frequency with which the patient contact relatives and the feeling of social connection. - (Better) Quality of conversations. To evaluate whether the robot can improve the quality of conversations, we can invite both patients and relatives to do a survey and determine through observational research. - Reduce emotional stress. To evaluate whether the robot can reduce emotional stress, we can invite the patient to do a survey. - Better relative attitude. To evaluate whether the robot can positively improve relatives' overall attitudes, we can invite them to do a survey or examine their willingness to take the initiatives to interact with the patient. - Lack of privacy. We can tell the negative effect when the patient and relatives personal information is leaked from the healthcare scenario. But this is the worst case. - Irritation. The patient's negative emotions might be obvious through the analysis of their words and facial expression. - Misinformation. The misinformation can be detected by the number of information corrections that occur in the conversations. |
Benchmark | For each measurement, what are the benchmarks (criteria)? (i.e., what are desired values?) | - Memory accuracy. The patient can call relatives' names correctly in 80% of conversations. - Social connection. The patient contacts relatives three times a week and reviews daily news every morning. - (Better) Quality of conversations. The patient can mention previous experience and chat happily in 80% of conversations. - Reduce emotional stress. The patient self-reports his/her emotional status positively in the questionnaire. - Better relative attitude. Relatives self-report their positive attitude about chatting with the patient in the questionnaire or in informal chats with doctors. Besides, relatives are willing to contact the patient three times a week. - Lack of privacy. People other than the patient, relatives and the doctor in charge are able to access the information stored in the robot. - Irritation. When the robot reminds the patient of things he/she remembers two times a day, we can assume the patient will irritate. In other cases, the patient self-reports his/her irritation. - Misinformation. The information stored in the robot is not correct. |
Demonstration of AI-functionality | Can you describe how you could demonstrate that your AI function (s) achieve(s) the effects that you listed in the previous question? | - Memory accuracy. The effect will be achieved when the patient is able to accurately recognizes relatives with the robot's help. - Social connection. The effect will be achieved when the patient has contact with relatives frequently either on phone or virtually with the robot's help. - (Better) Quality of conversations. The effect will be achieved when the patient is able to talk about things mentioned in previous conversations. - Reduce emotional stress. The effect will be achieved when the patient expresses his/her stress-free status or is observed by relatives and doctors around. - Better relative attitude. The effect will be achieved when relatives are willing to interact with and take care of the patient. - Lack of privacy. The effect will be achieved when the information about both the patient and relatives is not securely stored and can easily be accessed by others. - Irritation. The effect will be achieved when the robot reminds the patient of things he/she still remembers frequently. - Misinformation. The effect will be achieved when the information stored in the robot is incorrect, such as misconceptions about a relative. |
