vCare Pilot phase in Italy “Open the curtain” is approved by the expert users focus group

While the vCare Pilot phase in Italy is finally about to begin, Casa Di Cura del Policlinico di Milano has organized an online workshop involving a panel of experts representing four hospitals, all of them recognized as Italian excellence in clinical research and practice.

The main objective was to collect external inputs before the launch of the Pilot phase. The pool of experts have approved the trial design. They however consider that the actual implementation of a rehabilitation@home service is still beyond the immediate horizon.

The four participating hospitals were:

  • IRCCS Ospedale San Raffaele, Department of Neurological Diseases;
  • IRCCS Humanitas Research Hospital, School of Physiotherapy, within the Nursing, Neuropsychiatry and Rehabilitation Sciences Department;
  • IRCCS Fondazione Don Gnocchi, Research and Innovation Department;
  • Instituto Prosperius Tiberino, Neurorehabilitation Department.


Here are the most relevant insights which emerged during the workshop:

First, the workgroup undersrood the need to involve the Italian regulatory agency, in order to standardize tele-rehabilitation prescriptions among the refunded programs for healthcare providers. Currently, refund policies of the National Health system still reflect the “traditional” rehabilitation, so reimbursement policies for digital services are shaped on the therapies performed in the ambulatory setting and not yet in the residential setting.

Second, from a scientific perspective, they acknowledged that literature recognizes the positive effects of tele-rehabilitation, pointing out in particular an increased adherence, when therapy is achieved in a game-based setting, They however mentioned that the entire Italian scene still lacks of a clear objective investigation regarding who could be the ideal “end user” who could benefit of services like the ones proposed by vCare. Starting from here, the pilot phase should help to answer a number of key questions such as:

  • Which subjects are more responsive to tele-rehabilitation services ?
  • What is the clinical and social profile of those subjects ?
  • When is the best moment to take them in charge ?
  • Which service configuration is best adapted to the vCare stroke use-case ?


Third, there is no clinical protocol evidence regarding the validation of sensors and technological devices able to effectively assess the outcome measures that the vCare project intends to reach. While the need remains for collecting large amounts of data in the next years, through further multicentric RCTs and research projects, already now the vCare pilot phase can provide a first, significant “real life” experience.

Fourth, machine learning and predictive instruments are considered the biggest challenge to support the clinical decision-making process, especially if corroborated by a huge amount of input and output structured data. At the current state of the art, automated therapeutic decisions are not yet integrated in clinical practice, as this is still an experimental stage, but it is a promising perspective.

Finally, the support of a virtual assistant might be more appropriate for supplying information to caregivers, rather than to directly coach the patient.

Don’t worry!!! vCare is following the right direction as pioneers of a new healthcare approach, built on evidence-based pathways and scientific validation. Stay tuned for the beginning of the Pilot phase, as soon as when we’ll know more…

Design and implementation of the vCare Avatar-based Virtual Coach

The vCare patient User Interface (UI) is realized as an Android application running on a tablet device. The UI supports different representation and interaction modalities, including the human-like avatar, the vCare Virtual Coach (VC).

The VC is a ubiquitous part of the UI that assists and guides the patient through their rehabilitation program. The goal of the human-like UI representation is to engage and motivate the patient to adhere to their rehabilitation plan and doctor’s advices in their home environment. The great benefit and advantage, compared to non-avatar-based interfaces, is that they can use human-like interaction styles such as verbal and non-verbal communication expressed by a recognizable human-like Avatar. Therefore, the Avatar conveys information verbally using real-time state of the art Text-To-Speech (TTS) technology along with predefined gestures, motions, and emotion expressions. Conversely, patients can respond and interact with the Avatar verbally via speech input, which is recognized using state of the art speech recognition technology, enabling more natural and human-like communication patterns to create a livelier user experience.

Design Workshop

A design workshop was conducted with technical and clinical project partners. The goal of this workshop was to define different aspects of the VC and the representation by an Avatar, like general characteristics, qualities, faults, and the interaction with patients. For this, two team exercises were performed. The first exercise was to draw/sketch the VC and to define its key character features: age, sex, weight/size, qualities, faults, and the name. The second exercise was to describe a typical day from the VC’s point of view, i.e., how to interact with the patient. The collected data during the creative workshop was qualitatively analysed as follows.

Figure 1: Design workshop Avatar and scenery drawings/sketches

General characteristics of the Virtual Coach

One outcome of the workshop was that the VC should have an age between 30 and 40: the idea is that the VC should show both, to be an expert in the rehabilitation sector and an expert in the digital world. The VC is mainly seen as a female. This aspect is related to caring behaviours and feelings. Furthermore, behind the aspect of the VC, users should recognise both a physician, which is an expert in diseases and rehabilitation programmes and a “gym” trainer, which is an expert in physical activity and healthy behaviours, such as sport or diet. Figure 1 shows Avatar and scenery drawings/sketches produced at the design workshop.

Figure 1: Design workshop Avatar and scenery drawings/sketches

Qualities, faults, and interaction with patients

Other aspects such as the qualities of the VC, its faults and how the interaction with the patients can look like were also discussed in that workshop. Qualities were categorized in relational skills and professionalism, with associated attributes mentioned like patience, friendly, open-smile and punctual, precise or authoritativeness.

Faults were used to better define the VC’s characteristics and to try to balance all the aspects, avoiding the risk of negative perceptions/evaluations by users. The VC shouldn’t seem to be perceived as being too proud, too strict or directive and shouldn’t exhibit bad human characteristics like greed.

With regards to a typical interaction between the VC and the patient, it was found that both “push” and “pull” mechanisms can be implemented, meaning that an interaction can be started by either party. When activities are planned, an interaction should consist of “Before Activities”, “During Activities” and “After Activities”. In “Before Activities”, the VC could send alerts according to the patient’s agenda, to remind the him or her to follow his or her plans. “During Activities”, such as motor exercises, the VC should support the performance of the patient by giving feedback, inciting, giving suggestions etc. “After Activities”, the VC should give positive feedback when activities are completed by the patient, e.g., “Bravo!”, “Well done!”.

The outcomes of the design workshop provided valuable input for the design of the Avatar look and feel. The general characteristics were used as guidelines regarding the age of the Avatar but also for the outfit and general appearance.

Technical realization

The affective Avatar is implemented following AIT’s Avatar Generation Toolchain (AGT). Figure 2 illustrates the overall architecture of the AGT.

Figure 2: Overall Architecture of the AGT.

The AGT consists of six main components (3D modelling, animation, audio processing, rendering, video generation, video delivery) classified in three different processing steps (pre-processing, processing and the optional post-processing). The pre-processing part defines the avatar’s character and appearance as well as the domain and scenario-specific setting including the avatar’s movements. The processing part is responsible for the audio processing and for the run-time animation and rendering. This part is influenced and controlled by internal and external parameters (“controlling parameters”; e.g., the text which has to be spoken, the avatars emotion or motion). The post-processing part is an optional part and just required in situations where a live rendering is either not possible or not applicable (e.g., if the avatar needs to be embedded in existing services or specific Web applications).

In vCare, the avatar component uses the Unity3D game engine for rendering the Avatar in an Android App. The Avatar’s characteristics can be adjusted to the current context to e.g., express emotions by varying facial expressions, varying voice intonations using Speech Synthesis Markup Language (SSML) and different animations for the Avatar’s body posture and gestures.

Evaluation and validation

The Avatar evaluation and validation will be part of vCare’s outcome measurements at the end of the pilot phase using TAM (Technology Acceptance Model) score, SUS score (System Usability Scale) and Qualitative UEQ (User Experience Questionnaire).

Avatar and UI impressions

Figure 3 shows the current male and female characters, resembling the design workshop’s input like the age or the recognition as a physician based on the outfit. Figure 4 shows the current UI implementation with the Avatar in the top left corner, some navigation elements below in the bottom left area and the main information area on the right side.


Figure 3: Male and female VC designs

Figure 4: Screenshot of the UI showing the Avatar in the top left corner and patient observations on the right-hand side.