Machine learning in vCare

Home-based rehabilitation is effective only when the patient gets truly engaged in the different suggested activities – not only on a short-term basis but with the ambition to support learning, behaviour adaptation and skills development over a long period of time. Meaningful planning together with responsive feedback and practice are the key determinants which improve the Quality of Life (QoL) of the patient supported by a home-based rehabilitation programme. Home rehabilitation offers an interesting perspective in term of cost-effectiveness when compared to traditional care provided by the hospital, provided of course that the clinical results are at least similar.

Successful care rehabilitation at home, that is aligned to the user’s needs without constant assistance by medical staff, has several requirements. It needs user-friendly coaching or guidance, an assessment with real-time feedback strategy, and continuous engagement. That can be achieved when the therapist or coach is personalised, and provides assistance based on the user’s needs and personal clinical pathways. Analysing, scheduling and providing pertinent advice has to be supported by machine learning mechanisms to enable automation.

vCare identifies three core capabilities of the personalised home rehabilitation – system safety/-trust, system improvement, and system initialisation.

System initialisation: Recommendation systems have to be correctly initialised in order to retrieve at least average – better than random – quality recommendations in the beginning. A successful system initialisation is also referred to as a warm start, which might be based on manually curated rules or experience gained from a similar, already trained system. It is essential that the system provides the required minimal performance for all the diseases covered and the available clinical pathways.

The vCare project implements a hybrid approach: it is composed of two ways of working. As a baseline, vCare makes use of a manually curated set of expert rules, comprising safe adaptations of clinical pathways for a given patient. In addition, an imitation learning approach monitors live adaptations by experts and trains a machine learning-based system, which models the adaptation of clinical pathways as a Markov Decision Process[1].

System improvement: When using supervised machine learning algorithms, a system needs training data in the form of labelled examples that generalise novel, unseen situations. Activity recommendation does not, however, fit into the standard machine learning paradigm, where such training data is easy to obtain. The system needs to learn which activities work well for any given patient: this needs interaction. This interaction can be enabled by monitoring the vital parameters of patients to ensure health improvements, but also by undertaking targeted questionnaires to assess the patient’s subjective well-being.

Even if it is possible to gather sufficient experience to generate highly accurate decisions, the performance of machine learning-based systems often gradually deteriorates over time or exposes important situations where the system still fails. Reasons comprise data shift, where the available information changes over time, or basic lack of generalisation capabilities. Here, one could simply add new labelled examples to the system, but this process is not guided (i.e.. it might require large amounts of manual interventions) and does not guarantee a significant improvement.

At its core, vCare applies safe reinforcement learning based on the modelled Markov Decision Process, where actual patient feedback is used to adjust the clinical pathway in order to better represent the patient’s needs.

System safety & trust: In health care, it is highly important to guarantee system safety and trust. For the patient rehabilitation use case, system safety entails that the patient is never placed in situations which might negatively impact his/her health. Building trust means that the system is accepted by both physicians and patients. To enable patients’ trust, the system needs to be able to reason about its decision-making process in order to explain a certain recommendation.

vCare incorporates structured knowledge that has been formalised in a domain ontology (modelled with a Resource Description Framework). vCare augments this available set of expert rules with structured information in order to ensure correct retrieval and sustainable extension of the rule database. An important next step is to develop explainable machine learning algorithms, which enable transparency of pathway adaptations by visualising the used subset of available information.

[1] A Markov model is a stochastic method for randomly changing systems where it is assumed that future states do not depend on past states. These models show all possible states as well as the transitions, rate of transitions and probabilities between them.