Project homeStep 4: Claims
| Topic | Question | Answer |
Measurements | For each positive and negative effect listed in step 3, describe how you could evaluate (measure) whether they actually occur. | - Implicit feedback through usage patterns of patients. Measure using a mood graph with a threshold value to quantify a mood. Since the person is likely to have some sense of mobility and is in control of their choices, the system could understand the patient's (Georgina's) mood based on her choice of stories. - Explicit feedback from caregivers The system could ask the caregivers to enter a Yes/No for whether each task was performed. For example, did the patient take medicine after being reminded? Or did the patient eat their meal happily? - To find the dependency of the users on the system, the system could be taken down for a day or two. The caregiver (Eleana) could aid the patient instead and then answer questions on whether she was able to effectively perform tasks otherwise automated by the AI system. |
Benchmark | For each measurement, what are the benchmarks (criteria)? (i.e., what are desired values?) | - For the mood graph, if the values are between 1 and 10, we could keep a benchmark of around 5-6 so that the system improves its performance to adhere to the patient's preferences. While the patient's mental state is not always in control of the system, it could prove to be a stabilizing factor. - For the explicit feedback, we could set a benchmark of around 70-80% positive feedback, which would imply that the patient was able to perform 70-80% of the tasks successfully. (Scenario A) - To measure dependency, we could use the same explicit feedback but set a lower benchmark of 65-70% since we remove the system from the interaction. (Scenario B) |
Demonstration of AI-functionality | Can you describe how you could demonstrate that your AI function (s) achieve(s) the effects that you listed in the previous question? | - To demonstrate that the AI achieves the desired effects, we could plot the mood graph which hopefully shows a slightly increasing trend, above the threshold value. - To prove the usefulness of the system, we could compare the feedback given between Scenario A and B, and hopefully, show that the feedback is better with the AI. Ideally, the feedback is better with the AI but not to a very large extent since we do not want the patient to become overly dependent on the system. |
